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Preface 


Operations  of  the  Psychological  Research  Project  (Radar)  were 
carried  out  by  a  coordinated  research  group.  From  the  point  of  view 
of  recognizing  individual  contributions  to  this  report,  this  fact  has  a 
number  of  implications.  It  is  impossible,  for  example,  to  assign  in¬ 
dividual  responsibility  for  critical  stages  of  research  planning  which, 
for  the  most  part,  involved  group  consideration  and  decision.  Even 
in  project  development,  which  was  carried  out  by  research  teams,  es¬ 
sential  contributions  of  a  critical  nature  must  go  unidentified. 

With  these  qualifications,  an  effort  has  been  made  Jo  footnote  the 
names  of  those  persons  who  carried  central  responsibility  for  various 
areas  of  the  project’s  work.  In  addition,  the  writers  of  each  chapter 
have  been  named,  although,  here  again,  critical  contributions  from  the 
group  notably  altered  original  outlines  and  drafts. 

Beyond  this  it  did  not  seem  wise  to  attempt  in  the  text  individual 
recognition  for  what  was  so  effectively  a  cooperative  and  coordinated 
operation.  On  the  other  hand,  brief  note  should  be  taken  of  certain 
individual  services  which  were  fundamental  to  the  organization’s 
work  and  to  the  preparation  of  the  report.  For  example,  Capt.  H. 
Richard  Van  Saun  and  Sgt.  Albert  H.  Hastorf  did  invaluable  work 
in  research  coordination.  Staff  Sgt.  Bernard  C.  Sullivan  organized 
an  extensive  system  of  research  records  and  supervised  the  Project’s 
early  statistical  work.  Staff  Sgt.  Roland  E.  Johnston  carried  these 
same  responsibilities  during  a  later  period  when  extensive  IBM  an¬ 
alysis  was  being  carried  out.  For  a  short  but  critical  period  the  proj¬ 
ect  profited  from  the  statistical  services  of  Lt.  Sol  M.  Roshal  and  Capt. 
William  F.  Long. 

Basic  to  all  the  research  analysis  carried  out  by  the  project  was  its 
research  record  system.  Primarily  instrumental  in  its  development 
and  maintenance  were  Cpl.  Hyman  Sofer,  Sgt.  Samuel  D.  Morford 
and  Cpl.  Arlene  Babcock. 

Graphic  materials  employed  by  the  project  in  its  research  and  used 
in  this  report  were  the  work  of  Sgt.  Alfred  S.  Arnott. 

Ail  development  of  physical  measurement  instruments  and  main¬ 
tenance  of  testing  apparatus  was  the  primary  responsibility  of  Techni¬ 
cal  Sgt.  George  M.  Bollinger. 

Principal  credit  for  effective  administration  of  the  project’s  affairs 
during  the  major  period  of  its  work  goes  to  Sgt.  Harold  I.  Roth,  Staff 
Sgt.  Lester  I.  Foster,  and  Cpl.  John  D.  Hennessy.  During  the  emo- 
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tionally  trying  stages  of  final  editing,  mimeographing,  proofreading, 
and  assembling  this  report,  entire  responsibility  for  administration 
was  turned  over  to  Capt.  Gabriel  D.  Ofiesli.  Coordination  and  admin¬ 
istration  of  the  project’s  field  operations  were  carried  out  by  Lt.  Stuart 
Lottier  and  Lt.  Lewis  G.  Carpenter,  Jr. 

In  addition  to  the  aid  already  mentioned  the  editor  received  in¬ 
valuable  assistance  at  other  points  in  the  preparation  of  the  report. 
Lt.  Stuart  Lottier  read  all  chapters  from  the  point  of  view  of  improve¬ 
ment  in  style.  A  parallel  review  was  made  by  Cpl.  Harold  II.  Kelley 
from  the  point  of  view  of  technical  content.  The  exacting  task  of  re¬ 
viewing  the  report  for  adherence  to  certain  formal  conventions  estab¬ 
lished  for  the  aviation  psychology  research  series  ns  a  whole,  was 
carried  out  jointly  by  Sgt.  Ted  P.  Ivisciras,  Sgt.  Harold  I.  Roth,  and 
(’apt.  Gabriel  D.  Oficsh. 

Loyal  and  efficient  secretarial  service  during  the  period  of  the  proj¬ 
ect's  research  activities  was  rendered  by  Miss  Mary  Kingrea,  Mrs. 
Laura  "Winter,  Mrs.  Rose  Singer,  Mrs.  Virginia  Van  Satin,  and  Mrs. 
Mildred  Flanagan.  Mrs.  Margaret  Gage  was  invaluable  in  her  role  as 
research  librarian.  During  the  preparation  of  the  mimeographed  re¬ 
port  the  difficult  clerical  load  was  carried  by  Miss  Nora  Jenkins,  Miss 
Phyllis  Ashburn,  and  Mrs.  Christine  Glynn. 

In  addition  to  the  full-time  project  personnel,  Maj.  B.  von  Haller 
Gilmer  gave  constant  assistance  to  the  research  program  in  his  super¬ 
visory  capacity  at  Headquarters,  AAF  Training  Command.  Capt. 
Ike  H.  Harrison  served  as  a  valuable  consultant  on  navigational  and 
bombing  problems. 

Finally,  attention  should  bo  called  to  the  significant  contribution  of 
Col.  William  M.  Garland.  Such  success  as  had  been  achieved  in  psy¬ 
chological  research  in  the  radar  training  program  is  due  in  no  small 
measure  to  his  vision  and  to  his  continued  assistance  and 
encouragement. 

Langley  Field,  Va.,  Stuart  W.  Cook 

March  1, 1946  Capt .,  A.  G. 


IV 


Contents 


Chapter 

Preface . . . 

1.  Scope  of  the  Report . . . . . 

2.  The  Radar  Oherver  in  the  Army  Air  Forces . 

Development  of  Radar . . 

Airborne  Radar  in  Combat  Operations . . 

Training  of  Radar  Observers  in  tho  United  States . 

Summary . . . . . . . . 

3.  Survey  of  Research . . 

National  Defense  Research  Committee-Project  SC-70, 

NS-146 . . 

Aircrew  Evaluation  and  Research  Detachment  No.  1 . 

Selection  of  Radar  Observers . 

Psychological  Research  Project  (Radar) . . . 

Activation  of  tho  Project . . . 

Research  Objectives  and  Priorities . 

Survey  of  Project  Research . 

Personnel-. . 

Summary. . . . . . 

4.  Jod  Description  and  Analysis . 

General  Considerations . 

Reason  for  Describing  a  Job . 

Methods  of  Describing  a  Job . 

Basis  of  Present  Definition  of  Radar  Observer’s  Job.. 
Job  Description  and  Analysis  of  Radar  Observer  Students 

in  Training . . 

Extent  of  Job  Description  and  Analysis . 

Introduction  to  Air  Borne  Radar  Equipment . 

Set  Operation:  Job  Description . 

Set  Operation:  Job  Analysis . 

Scope  Interpretation:  Job  Description . 

Scopo  Interpretation:  Job  Analysis . 

Navigation:  Job  Description.. . . . 

Navigation:  Job  Analysis . ‘ . . 

Bombing:  Job  Description — . . 

Bombing:  Job  Analysis . . . . . 

Summary . . . . . 

V 


Pay! 

Ill 

1 

5 

5 

6 
8 

10 

11 

11 

12 

13 

14 
H 

15 

16 
18 
19 
21 
21 
21 
21 
24 

26 

26 

27 

29 

31 

32 

34 

35 

40 

41 
46 
40 


Chapter  Pa  ft 

4.  Job  Description  of  the  Radar  Observer  in  Combat _  47 

Sources  of  Information . . .  47 

European  Theater _ _  47 

Pacific  Theater . . 49 

Job  Analysis  of  the  Radar  Observer  in  Combat -  51 

Set  Operation _ _  51 

Scope  Interpretation _ 52 

Navigation _ _ 52 

Bombing _ 52 

Summary. . _ 53 

The  Job  of  the  Radar  Observer  in  tho  Futuro. . .  53 

Trends  in  Equipment  Development _  53 

Indicated  Changes  in  the  Radar  Observer’s  Job .  54 

Summary _ 54 

5.  The  Development  of  Printed  Proficiency  Tests _  57 

Introduction. . 57 

Uses  of  Standardized  Proficiency  Tests  in  Radar  Observer 

Training . 58 

General  Procedures  Used  in  Developing  Printed  Tests _  60 

Preliminary  Drafts . . 60 

Review  and  Criticism . . 61 

Test  Format . 62 

Time  Limits .  62 

Scoring  Formula .  62 

Bases  for  Revisions. .  63 

Standardization  of  Test  Administration .  64 

Difficulties  Experienced  in  tho  Use  of  Standardized  Pro¬ 
ficiency  Tests . 65 

Description  of  Typical  Printed  Tests  Used  in  the  Radar 

Training  Program . . 66 

Set  Operation  Intermediate  Test .  66 

Radar  Navigation  Intermediate  Test .  69 

Radar  Bombing  Intermediate  Test . .  72 

Radar  Intelligence  Intermediate  Test .  75 

Final  Test  I _ 76 

Navigation  Proficiency  Test . .  88 

Summary.  _ _ 90 

6.  Standardized  Performance  Checks . .  95 

Introduction - 95 

General  Procedure  Used  in  Developing  Performance 

Checks . 96 

Arrangement  and  Format . 97 

Examiner  Training.. . 98 

Check  Revisions . . . - . .  98 


VI 


Che, <  ft  pag, 

6.  Standardized  Bench  Set  Performance  Checks. .  99 

The  Bench  Set  and  the  Student's  Task . . .  99 

Bench  Check  Construction . 101 

The  Typical  Bench  Set  Check .  101 

Types  of  Bench  Check  Items . .  102 

Bench  Check  Administration . 102 

Bench  Check  Revisions . .  103 

Bench  Check  Statistical  Findings . .  103 

Standardized  Supersonic  Trainer  Performance  Checks _  106 

The  Supersonic  Trainer  and  tho  Student's  Task .  106 

Supersonic  Check  Construction.  . . . .  107 

The  Typical  Supersonic  Check . . : .  107 

Types  of  Supersonic  Check  Items .  108 

Supersonic  Check  Administration . .  ill 

Supersonic  Check  Revisions . .  111 

Supersonic  Check  Statistical  Findings .  112 

Standardized  Aerial  Performance  Checks .  116 

The  Airborne  Set  and  the  Student's  Task .  116 

Aerial  Check  Construction . . .  116 

Tho  Typical  Aerial  Check .  117 

Types  of  Aerial  Check  Items .  117 

Aerial  Check  Administration .  118 

Aerial  Check  Revisions .  120 

Aerial  Check  Statistical  Findings. .  120 

Tho  Student  Rating  Research  Form .  122 

Summary .  123 

7.  TriE  Measurement  of  Performance .  127 

Tho  Structure  of  Performance  Check  Items. . .  127 

Background  for  an  Analysis  of  Sources  of  Unreli¬ 
ability . 127 

General  Structure  of  Items . . . 127 

Student  Performance.  __ . 128 

Evaluation  of  Student  Performance .  129 

Discrete  and  Interdependent  Items .  131 

Direct  and  Indirect  Measurement.. .  132 

Reliability  of  Performance  Check  Scores .  133 

Reliability  With  Which  Performance  Reflects  Profi¬ 
ciency _ _ _ _ _ -  133 

Reliability  of  Student  Performance  Data .  135 

Reliability  of  tho  Evaluation  of  Student  Performance.  135 

Tho  Measurement  of  Reliability . 136 

Validity  of  tho  Performance  Check .  138 

Curriculum  Validity . 139 

Predictive  Validity . .  139 

vn 


it U J,l|! f > V'7^r;T w; r f r ■ 


WU'IJIJLU 


flMM.I!1.".! 


i 


i 


Chapter  Page 

7.  Measures  of  Proficiency  Within  the  Same  Stage  of  Train¬ 

ing  ns  Criteria . . . —  140 

Summary _ 142 

8.  INTERRELATIONS  OF  PROFICIENCY  MEASURES -  143 

The  Variables  and  the  Sample _  144 

Variables _ 144 

IntercoiTclations  of  Total  Scores  for  Twelve  Proficiency 

Measures _ _ _  147 

Correlations  Between  Printed  Tests  and  Performance 

Checks  Measuring  Similar  Skills _ _ 148 

Test  Scores  and  Performance  Check  Total  Scores _  149 

Test  Scores  and  Performance  Check  Part  Scores _  150 

Proficiency  Stanines _  151 

Correlations  Between  Pairs  of  Tests  and  Between  Pairs  of 

Performance  Checks  Measuring  Similar  Skills _  152 

Correlations  Between  Performance  Checks  Measuring 

Similar  Skills . . . .  152 

Correlations  Between  Tests  Measuring  Similar  Skills.  154 
Correlations  Between  Ratings,  Performance  Checks,  and 

Final  Tests . . . . . . . .  156 

Correlations  Between  Navigation,  Set  Operation,  and 

Bombing  Skills .  157 

Part  Against  Part  of  the  Same  Test .  157 

Part  Against  Part  of  the  Same  Performance  Check...  158 

Test  Against  Test . . .  158 

Performance  Check  Against  Performance  Check _  159 

Test  Against  Performance  Check .  159 

Correlations  Between  Part  Scores  of  Final  Test  1 .  161 

Related  Reports  From  the  Aviation  Psychology  Program.  163 

Printed  Tests  Against  Aerial  Performance  Checks _  163 

Correlations  Between  Printed  Tests . .  164 

Summary . . . . .  165 

9.  The  Circular  Error  in  Radar  Observer  Training _  167 

Introduction . . . .  167 

Recording  and  Measuring  Bombing  Hits .  167 

Photographs  of  Actual  Bomb  Drops .  168 

Photographs  of  Simulated  Bomb  Drops . .  168 

Recording  by  Gun-Laying  Radar  (SCR-584) . 170 

Bombing  Proficiency  Scores _ _ _  171 

Description  of  Samples  Used  in  Circular  Error  Studies _  171 

Boca  Raton . . . . .  171 

Victorville.. _ _ _ _  172 

Langley  Field. . . . .  172 

vra 

/  i 


■ 


*■  ,n  *< «  ; 


Chapter  Page 

9.  The  Reliability  of  Average  Circular  Error _  173 

Evaluation  of  Methods  of  Determining  Reliability _  175 

Statistical  Studies  of  Reliability . .  177 

Discussion  of  Reliability  Results _ 183 

Comparison  of  Results  With  Studies  of  Visual  Bomb¬ 
ing - 183 

Variable  Factors  in  Radar  Bombing  Missions _  134 

Improving  the  Reliability  of  Average  Circular  Error.  189 
The  Learning  Curve  for  Bombing  in  Radar  Observer 

Training . 191 

Relation  Between  Bombing  Practice  and  Circular 

Error _ 191 

The  Learning  Curve  for  Radar  Bombing .  192 

Relationship  Between  Average  CE  and  Other  Proficiency 

Measures _ _ 197 

Relation  Between  Final  Course  Grade  and  Average 
CE,  Computed  With  and  Without  Extreme  Misses.  197 
Relation  Between  Finai  Course  Grade  and  CE,  With 
and  Without  Bombing  Practice  Partialled  Out....  198 

Relation  Between  Phase  Grades  and  Circular  Error..  199 
Relation  Between  Single  Proficiency  Measures  and 

CE . . 200 

Systematic  Error  in  Radar  Bombing  Training:  The  Range 

Over  Error .  201 

Equipment  Sources .  201 

Psychological  Sources . .  203 

Controlling  the  Range  Over  Error .  205 

Summary . 205 

10.  History  of  Radar  Observer  Selection .  211 

Selection  Research  by  Project  SC-70,  NS-14G  National 

Defense  Research  Committee .  211 

Tests . .  212 

I 

Selection  Research  by  the  Aircrew  Evaluation  and  Re¬ 
search  Detachment  No.  1 . 216 

Student  Selection  by  Psychological  Research  Project 

(Navigator) . 223 

Student  Selection  by  Headquarters,  AAh*  Training  Com¬ 
mand . 224 

Summary . 227 

11.  Validation  of  Selection  Tests  for  Radar  Observer 

Training . .  229 

|  Introduction . .  229 

! 


i 


DC 


Chapltt  payt 

11.  Validation  Study  I _ 230 

Variables _ 230 

Validation  Criteria _ _ _  231 

Composition  of  Validation  Samples _  233 

Validation  Statistics _  234 

Validation  Study  II _ 237 

Variables _ 237 

Validation  Criteria _  242 

Composition  of  Validation  Samples _  245 

Validation  Statistics _  246 

Multiple  Regression  Statistics _  251 

Indicated  Validity  of  Factors . . 257 

Evidence  for  Factor  Validity.. _ _  258 

Summary . . . 268 

12.  An  Evaluation,  With  Suggestions  for  Future  Re¬ 

search . 271 

Job  Analysis  and  Selection-Test  Research _ 271 

Proficiency  Measurement . 275 

Radar  Bombing  Circular  Error . 278 

Instructor  Selection  and  Evaluation . . 279 

Training  Resoarch  . .  280 

Appendix  A .  283 

Glossary  of  Technical  Terms . 324 

Index .  333 


X 


CHAPTER  ONE 


Scope  of  tlie  Report1 


This  report  describes  the  psychological  research  conducted  in  rela¬ 
tion  to  the  selection  and  training  of  radar  observers  (bombardment)5 
in  the  Army  Air  Forces.  During  the  first  2  years  of  the  war,  opera¬ 
tions  of  heavy  bombers  were  conducted  without  radar s  aids.  Late  in 
1913,  satisfactory  airborne  equipment  became  available  in  small  quan¬ 
tities.  A  year  later,  production  of  radar  sets  had  accelerated  to  such 
a  point  that  it  was  necessary  to  initiate  a  large  scalo  training  program. 
The  research  activities  described  in  this  report  took  place  largely  ha 
connection  with  this  training  program. 

The  report  begins  with  an  account  of  tho  use  of  radar  as  a  device 
for  blind  bombing  and  navigation  (ch.  2).  Tho  account  includes  a 
brief  explanation  of  the  basic  principles  on  which  radar  operates,  in¬ 
cluding  tho  use  of  high  frequency  pulsating  radio  waves  and  tho  trans¬ 
mission  and  reception  of  these  waves.  A  few  of  tho  major  technical 
developments  are  noted,  such  as  tho  early  experiments  with  radar  by 
naval  scientists.  Tho  emphasis  is  placed  on  the  development  of  air¬ 
borne  radar  as  a  strategic  and  tactical  weapon  in  World  War  II,  start¬ 
ing  with  tho  first  aircraft  warning  sets  and  culminating  in  tho  uso  of 
airborne  radar  in  bombing  and  navigation  in  tho  European  and  Pacific 
Theaters.  Finally,  a  survey  is  made  of  tho  development  of  radar 
observer  training  both  in  overseas  installations  and  in  thoso  located 
in  tho  continental  United  States. 

Chapter  3  presents  a  chronology  of  research  activities  to  servo  ns 
a  framework  for  more  detailed  descriptions  in  later  chapters.  Tho 
major  research  undertakings  of  each  of  tho  three  organizations  which 
accomplished  psychological  research  on  the  radar  observer  aro  enu¬ 
merated  and  their  interrelations  pointed  out.  The  survey  begins  with 
the  research  done  by  the  Radar  Project  established  in  February  1943, 
at  Camp  Murphy,  Fla.,  by  the  National  Defense  Research  Committee 


’Written  by  Sgt  Albert  H.  Hastorf. 

’The  full  title  of  the  aircrew  specialist  who  operated  rndnr  as  an  aid  to  boroblntf 
and  navigation  In  heavy  bombardment  aircraft  of  tho  Army  Air  Forces  was  radar  observer 
(bombardment).  For  purposes  of  brevity,  the  shorter  title,  radar  observer,  la  ueed 
throughout  tbla  report. 

•A  definition  of  this  a.  J  other  terms  requiring  explanation  la  Included  In  a  glOMiary 
to  be  found  at  tho  end  of  the  report 
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of  the  Office  of  Scientific  Research  and  Development  (Project  SC-70, 
NS-MG).  Next  presented  are  the  research  activities  of  the  AAF 
Aircrew  Evaluation  and  Research  Detachment  No.  1  on  the  selection 
of  radar  observers  at  the  radar  training  station  operated  by  the  Eighth 
Air  Force.  Finally,  there  is  given  a  description  of  the  work  of  the 
Psychological  Research  Project  (Radar).  This  project,  officially 
established  on  1  December  1911,  concentrated  its  efforts  on  the  psy¬ 
chological  problems  encountered  by  the  AAF  Training  Command  in 
selecting  and  training  radar  observe i  •  within  the  continental  United 
States. 


The  major  part  of  chapter  1  consists  of  a  job  description  and  analysis 
of  tho  task  of  the  radar  observer  based  upon  observations  of  ground 
and  aerial  training.  A  second  section  discusses  job  requirements  in 
combat.  Material  for  this  discussion  was  obtained  from  personal  in¬ 
terviews  with  combat  experienced  radar  observers  and  from  other 
reports  from  the  combat  theaters.  The  combat  analysis  differentiates 
the  activities  of  the  radar  observer  in  the  European  and  in  the  Pacific 
Theaters.  Finally,  there  is  a  statement  of  probable  requirements  lor 
the  radar  observer’s  task  in  the  future;  these  predictions  are  based  on 
current  knowledge  of  the  technical  advances  made  in  the  development 
of  new  airborne  radar  equipment. 

Chapters  5,  G,  7,  and  8  deal  with  profioicncy  criteria  developed  for 
tho  radar  observer  training  program  by  tho  Psychological  Research 
Project  (Radar).  A  battery  of  11  printed  proficiency  tests  and  per¬ 
formance  checks  was  used  throughout  training  for  evaluation  of 
student  proficiency.  Chapter  5  discusses  the  proficiency  tests,  chapter 
G  tho  performance  checks,  chapter  7  general  problems  in  the  measure¬ 
ment  of  performance,  and  chapter  8  the  interrelationships  between 
various  proficiency  measures. 

Tho  plan  for  proficiency  test  construction  called  for  intermediate 
tests  in  each  of  the  major  curriculum  divisions  and  one  final  compre¬ 
hensive  test.  Tho  tests  described  in  chapter  5  represent  each  of  these 
types  of  test.  A  brief  account  is  given  of  the  methodology  of  test  con¬ 
struction,  problems  encountered  in  the  standardization  of  test  admin¬ 
istration,  and  tho  difficulties  in  test  construction  resulting  from  an 
unstandardized  curriculum.  Among  tho  descriptive  materials  will 
be  found  sample  items,  means,  standard  deviations,  and,  wherever 
available,  reliability  coefficients. 

Chapter  G  presents  a  discussion  of  the  three  major  groups  of  per¬ 
formance  checks  developed:  Tho  bench  set  trainer  checks,  the  super¬ 
sonic  trainer  checks,  and  tho  aerial  checks.  A  statement  is  made  of 
the  rationale  for  the  development  of  theso  checks.  Their  construction 
is  described,  including  tho  selection  of  behavior  to  bo  evaluated  and 
tho  development  of  items  and  of  format.  Sample  items  and  statisti- 
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cni  data  are  included.  Also  described  in  chapter  6  are  the  major 
steps  in  the  training  of  examiners  to  administer  the  checks. 

Chapter  7  discusses  the  validity  and  reliability  of  performance 
measures.  A  distinction  is  made  between  predictive  validity  and 
curriculum  validity,  which  holds  a  measure  to  be  valid  if  it  furnishes 
a  comprehensive  test  of  achievement  for  a  given  area  of  instructional 
material.  Types  of  criteria  against  which  predictive  validity  may  be 
determined  are  described.  As  a  background  for  the  discussion  of 
reliability,  performance  check  items  arc  analyzed  in  terms  of  two 
components:  The  student’s  performance  and  the  examiner  evalua¬ 
tion  of  that  performance.  The  reliability  of  different  types  of  items 
is  evaluated  with  reference  to  measurement  errors  associated  with  these 
two  components.  Statistical  techniques  appropriate  to  the  measure¬ 
ment  of  performance  check  reliability  are  reviewed. 

Chapter  8  analyzes  the  interrelations  of  the  proficiency  measures 
described  in  chapters  5  and  G.  Among  the  matters  discussed  is  the 
relation  of  proficiency  test  scores  to  scores  on  performance  checks 
measuring  the  same  skills.  The  findings  are  applied  to  the  general 
problem  of  the  relationship  between  verbal  knowledge  and  actual  job 
performance  and  the  question  of  substituting  print*.  1  proficiency  tests 
for  performance  checks.  Another  section  of  the  chapter  is  devoted  to 
the  interrelations  of :  (1)  Performance  checks  which  measure  similar 

skills,  and  (2)  proficiency  tests  which  measure  similar  skills.  Also, 
evidence  is  presented  as  to  the  degree  of  relationship  between  three 
areas  of  radar  observer  skill:  Navigation,  bombing  and  set  operation. 
A  comparison  is  made  between  the  statistical  findings  reported  in  this 
chapter  and  parallel  findings  of  AAF  psychological  research  projects 
working  on  proficiency  measurement  in  bombardier  and  navigator 
training. 

Chapter  9  analyses  the  bombing  error  of  radar  observer  students 
in  training  within  the  continental  United  States.  The  chapter  in¬ 
cludes  a  description  of  alternate  methods  for  scoring  the  amount  of 
bombing  error.  The  reliability  of  the  camera  bombing  circular  error 
made  by  students  at  three  training  schools  is  presented.  Data  nro 
presented  also  on  the  reliability  of  actual  bomb  drops;  however,  these 
arc  available  from  one  training  school  only.  Types  of  variable  errors 
contributing  to  unreliability  are  discussed  and  suggestions  are  made 
for  increasing  reliability.  Correlations  are  given  between  circular 
error  and  certain  of  the  ground  and  aerial  proficiency  measures  de¬ 
veloped  by  the  Psychological  Research  Project  (Radar).  Also  dis¬ 
cussed  is  the  relationship  between  amount  of  practice  and  circular 
error.  Finally,  a  constant  error  evident  on  most  student  bombing 
missions  is  analyzed  at  length. 

Chapters  10  and  11  discuss  selection  research  in  radar  observer 
training.  Chapter  10  gives,  first,  an  historical  account  of  selection 
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research  prior  to  the  establishment  of  the  Psychological  Research 
Project  (Radar),  and,  second,  an  account  of  the  methods  through 
which  the  selection  of  radar  students  was  actually  accomplished.  The 
chapter  reviews  the  selection  research  accomplished  by  the  Radar 
Project  (SC-70,  NS-MG)  of  the  National  Defense  Research  Commit¬ 
tee  and  the  Air-crew  Evaluation  and  Research  Detachment  No.  1  of 
the  AAF  Aviation  Psychology  Program.  Tests  developed  by  each  of 
these  groups  are  described.  Results  are  reported  for  a  validation 
study  carried  out  by  the  latter  organization. 

Chapter  11  is  concerned  with  selection  test  research  conducted  by 
the  Psychological  Research  Project  (Radar).  Two  validation  studies 
were  completed,  each  based  upon  a  sample  of  bombardiers  and  a 
sample  of  navigators.  The  first  study  validates  the  Air-crew  Classifi¬ 
cation  Battery  and  the  Radar  Observer  Selection  Battery  against  a 
course  grade  determined  by  the  training  schools.  The  second  study 
validates  not  only  tests  from  these  two  batteries  but  also  experimental 
psychomotor  and  printed  tests.  In  the  second  study,  the  criterion  for 
the  bombardier  sample  was  a  course  grade  computed  by  the  Project 
on  the  basis  of  standardized  proficiency  measures.  The  variables  for 
the  navigator  sample  were  validated  against  course  grades  determined 
by  the  school  and  also  against  radar  bombing  error.  In  addition  to 
validity  coefficients,  multiple  correlation  statistics  are  presented. 
Chapter  11  also  includes  a  discussion  of  empirically-determined 
attributes  of  the  successful  radar  student 

Chapter  12  evaluates  the  research  accomplished  to  date  and  presents 
a  prospectus  for  future  investigation. 

Tho  report  has  one  appendix  and  a  glossary  of  technical  terms; 
appendix  A  consists  of  descriptions  of  tho  selection  tests  validated 
by  Psychological  Research  Project  (Radar). 
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CHAPES  TWO 


The  Radar  Observer  in  the  Army 

Air  Forces1 


Radar  was  one  of  the  outstanding  technical  developments  of  World 
War  II.  Used  in  detection  and  warning,  aircraft  interception,  sub¬ 
marine  hunting,  bombing,  navigation,  fire  control,  and  blind  landing, 
it  proved  itself  important  in  both  offensive  and  defensive  action.  The 
purpose  of  this  chapter  is  to  describe  the  use  of  radar  by  the  Army 
Air  Forces  as  an  aid  to  aerial  bombing  and  navigation.  The  chapter 
will  include  an  account  of  the  training  of  the  radar  observer  with 
emphasis  upon  the  program,  within  the  AAF  Training  Command. 

DEVELOPMENT  OF  RADAR 

Radar  is  a  contraction  of  the  words  Radio  Detection  and  Ranging. 
Its  basic  principle  is  that  of  the  echo,  with  high  frequency  radio  waves 
substituted  for  sound  waves.  It  is  a  familiar  fact  that  a  sharp  noise 
or  a  shout  near  a  cliff  or  high  wall  will  bo  returned  as  an  echo.  The 
longer  it  takes  the  echo  to  return,  the  greater  is  the  distance  to  the 
reflecting  obstacle. 

In  radar,  short  pulses  of  radio  energy,  traveling  at  180,000  miles 
per  second,  arc  sent  out.  If  an  echo  returns,  the  radio  pulse  has 
reached  some  reflecting  object.  By  measuring  the  lapse  of  time,  the 
distance  to  the  object  can  bo  found,  and  by  determining  the  direction 
from  which  the  echo  returns,  the  bearing  of  the  object  can  be 
determined. 

Radar  was  an  outgrowth  of  radio  research  conducted  over  a  period 
of  many  years.  The  first  step  in  its  development  cumo  in  1922  when 
experimenters  were  working  with  high  frequency  transmitting  and 
receiving  equipment  at  the  Naval  Aircraft  Radio  Laboratories.3  On 
one  side  of  the  Potomac  River  they  had  installed  a  transmitter  and, 
on  the  other  side,  a  receiver  which  converted  the  reflected  radio  energy 
to  visual  form  on  an  oscilloscope  screen.  They  noticed  that  a  ship 
passing  between  the  transmitter  and  the  receiver  interfered  with  ro- 

1  Written  by  Sgt.  Albert  II.  Hnstorf. 

’"Story  of  Radar”  prepared  under  tlio  direction  of  the  Commanding  General,  A  AS" 
Eastern  Technical  Training  Command,  by  Undlo  Publication*  Division,  April  1015,  p,  & 
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ccpf.ion.  Realizing  tlic  possibilities  of  this  discovery,  they  continued 
investigation  of  the  phenomenon  and  found  that,  not  only  would  a 
ship  interfere  with  radio  waves,  but  that  iL  would  reflect  them  as  well. 
'I'll is  suggested  the  desirability  of  placing  the  transmitter  and  receiver 
in  the  same  place.  Iiy  1930,  rosea r  1  had  developed  to  the  point  where 
the  presence  of  surface  vessels  hidden  by  fog,  smoke,  or  darkness  could 
bo  detected. 


As  research  cont  inued,  methods  of  aircraft  detection  were,  developed. 
It  was  found  that  an  aircraft  passing  between  a  transmitter  and  re¬ 
ceiver  also  set  up  an  interference  pattern.  By  1934,  both  the  direc¬ 
tion  and  distance  of  the  aircraft  could  be  determined  accurately.  In 
1938,  radar  sets  had  been  installed  in  some  naval  ships.3 


AIKBOHNE  RADAR  IN  COMBAT  OPERATIONS 

The  first  important  wartime  application  of  airborne  radar  came  in 
the  air  battle  over  Great  Britain.  In  that  battle,  ground  radar  warn¬ 
ing  stations  detected  approaching  enemy  planes  and  informed  fighter 
pilots  by  radio  of  the  enemy’s  approximate  position.  The  fighter 
pilots  then  proceeded  to  make  contact  visually.  At  night,  however, 
or  in  the  foggy  weather  often  encountered  over  England,  visual  con¬ 
tact  was  difficult  and  often  impossible.  To  remedy  this  situation  the 
RAF  turned  to  airborne  radar. 

RAF  fighter  aircraft  were  equipped  with  aircraft  interception  sets. 
Ground  radar  stations  directed  the  fighters  near  enough  to  the  enemy 
planes  so  that  these  short  range  sets  could  be  used.  It  is  reported  that 
in  one  24-hour  period,  radar-equipped  fighters  shot  down  232  aircraft 
at  a  cost  to  themselves  of  only  40  aircraft  and  12  pilots.4 

Tho  next  major  development  in  the  use  of  airborne  radar  took  placo 
in  tho  antisubmarine  campaign.  The  fundamental  type  of  search 
radar  set  was  air-to-surfnce-vessel  equipment,  designated  ASV.  The 
earliest  widely  used  set  of  this  type  was  a  long-wave  set.  This  was  soon 
replaced  by  more  accurate  microwave  equipment.  Another  import¬ 
ant  development  on  recent  ASV  sets  was  that  of  the  plan  position  in¬ 
dicator,  or  PPI  scope,  which  presents  a  3G0°  picture  of  tho  area  over 
which  tho  aircraft  is  flying.  With  these  sets,  operators  are  able  to 
pick  up  land  targets,  convoys,  single  ships  and  surfaced  submarines 
at  longer  ranges  than  were  previously  possible. 

Radar  observer  equipment  consists  of  a  search  set  similar  to  ASV, 
a  precision  ranging  unit,  and  a  bombing  computer.  With  it,  cities  and 
other  targets  can  be  observed  and  bombed  from  high  altitudes  through 
a  complete  overcast.  A  city  appears  as  a  white  patch  on  tho  scope 
at  distances  up  to  about  100  miles.  As  the  aircraft  approaches  the 
city,  the  white  patch  takes  the  approximate  shape  of  the  city.  Bomb- 

*  Ibid.,  p.  2. 

*  Ibid.,  p.  3. 
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in"  can  be  accomplished  in  two  ways.  In  coordinated  bombing,  the 
radar  observer  gives  the  bombardier  information  with  which  he  syn¬ 
chronizes  the  bombsigliL  to  make  an  accurate  release  even  though 
(lie  target  is  not  visible.  In  direct  bombing,  the  radar  observer  makes 
the  release  independently  of  the  bombardier,  by  use  of  the  radar  set 
alone. 

Because  the  PPI  scope  presents  a  rough  map  of  the  area  below  the 
aircraft,  the  radar  set  is  a  very  useful  aid  to  navigation.  From  the 
scope,  the  radar  observer  can  locate  the  position  of  his  aircraft  by 
determining  its  direction  and  distance  from  landmarks  such  as  coast¬ 
lines,  lakes,  rivers,  mountains  and  cities.  lie  is  thus  able  to  establish 
his  position  even  though  visual  observation,  or  rad'o  contact  is 
impossible. 

The  joint  British  and  American  seizure  of  the  aerial  offensive  in 
the  summer  of  1913  gave  airborne  radar  its  first  opportunity  as  an 
offensive  weapon.  Earliest  use  of  radar  for  blind  bombing  and  navi¬ 
gation  was  by  pathfinder  crews  of  the  RAF.  The  AAF  flew  its  first 
radar  bombing  mission  in  September  of  1913,  using  British  equipment. 
Improved  equipment  was  soon  developed  at  the  Radiation  Laboratory 
of  the  Massachusetts  Institute  of  Technology,  and  mass  production 
of  radar  sets  was  begun  in  the  United  States.  A  number  of  different 
sets  were  used  by  the  AAF  but,  largely  because  of  the  highly  secret, 
nature  of  the  sets,  the  official  designations  were  rarely  used,  other  terms 
being  coined  for  radar  equipment,  observers,  and  operations.  RAF 
terms  include  II‘2S  and  Stinky;  AAF  terms  include  II2X,  Mickey, 
BTO  for  bombing  through  overcast,  and  Eagle  for  tho  AN/APQ-7  set. 

Previous  to  October  1943,  AAF  crews  had  been  trained  on  RAF 
radar  equipment.  In  October,  the  first  AN/APS-15  sets  arrived  from 
the  United  States  for  use  by  the  Eighth  Air  Force.  Four  aircraft 
were  equipped  with  these  sets  and  crews  were  assigned  to  fly  training 
flights  over  England  ard  the  North  Sen.  Eventually,  the  4S2d  Bomb 
Group  of  the  Eight  Air  Force  was  selected  ns  a  central  location  for 
such  aircraft.  From  this  group,  crews  and  aircraft  were  dispatched 
toffy  with  a  different  unit  of  the  Eighth  Air  Force  as  pathfinder  o  r  lead 
crews.  The  job  of  a  pathfinder  crew  was  to  lead  a  formation  of  bombers 
and  make  the  bomb  release  which  served  as  a  signal  for  the  entire 
formation  to  drop  its  bombs.  Personnel  of  the  lS2d  Group  trained 
additional  radar  observers.  As  more  radar  equipped  aircraft  were 
received  from  the  United  States,  the  group  begun  training  personnel 
for  permanent  assignment  to  other  units  equipped  with  pathfinder 
aircraft.  This  decentralization  made  it  possible  for  the  radar  crews 
to  operate  as  more  effective  .elements  of  the  group  to  which  they  wore 
assigned.  Previously,  efficient  team  work  had  been  difficult  because 
the  pathfinders  had  led  a  different  group  on  each  mission. 
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As  tile  need  for  trained  radar  observers  grew,  tile  Eighth  Air  Force 
established  a  specialized  training  school  with  a  formal  radar  observer 
course.  The  first  class  entered  on  25  February  1911  and  consisted  of 
42  navigators  and  bombardiers  selected  from  combat  groups  in 
England.  Each  class  received  a  month  of  ground  and  aerial  training 
in  radar  navigation  and  bombing. 

The  inflow  of  students  was  increased  so  that  by  May  of  191  1,  there 
were  75  students  in  a  class.  Approximately  two-thirds  of  each  class 
had  received  some  radar  training  in  the  United  States  prior  to  em¬ 
balming  for  England. 

Shortly  after  the  Eighth  Air  Force  began  extensive  operations  with 
radar,  the  Fifteenth  Air  Force  in  Italy  procured  its  first  radar  ob- 
servors.  Combat-experienced  navigators  and  bombardiers  had  been 
returned  to  the  United  States  to  receive  radar  training.  With  this 
personnel,  the  Fifteenth  Air  Force  first  began  combat  operations  w'ith 
radar  and  later  activated  its  own  training  school  for  radar  observers. 

The  use  of  airborne  radar  increased  rapidly.  In  November  1943, 
radar  equipment  made  it  possible  for  the  Eighth  Air  Force  to  fly  moro 
bombing  missions  than  in  any  previous  month.  Throughout  Novem¬ 
ber  and  December,  virtually  all  bombing  missions  were  led  by  radar 
equipped  pathfinder  crews.5  Gradually  radar  navigation  and  bombing 
techniques  were  refined  and  improved.  On  D-day,  all  heavy  bombard¬ 
ment  formations  were  led  by  radar  aircraft  and  all  coastline  targets 
were  bombed  with  the  aid  of  radar. 

In  the  Pacific  Theater,  in  the  early  stages  of  the  war,  radar  observers 
flew  in  B-24  bombers  on  sea-search  missions  against  Japanese  ship¬ 
ping.  Two  radar  sets  were  used :  The  radar  observer  operated  a  search 
set  such  as  the  SCR-717  or  AN/APS-15A,  and  the  bombardier  used  a 
radar  computer,  called  the  AN/APQ-5.  By  coordinating  the  opera¬ 
tion  of  these  two  sets,  the  radar  observer  and  the  bombardier  were 
able  to  locate  and  bomb  surface  vessels  with  great  accuracy. 

The  radar  observer  also  played  nn  important  role  in  the  bombard¬ 
ment  of  the  Japanese  home  islands.  During  the  last  months  of  the 
war  in  the  Pacific,  all  B-29  aircraft  were  equipped  with  either 
AN/APQ-13  or  AN/APQ-7  sets.  Early  operation  of  these  sets  was 
unsatisfactory  because  of  the  lack  of  adequately  trained  personnel. 
'I’ll is  situation  improved,  however,  early  in  1945  ns  the  expanded 
training  program  in  the  United  States  provided  a  greater  number  of 
qualified  radar  observers. 

TRAINING  OF  RADAR  OBSERVERS  IN  THE  UNITED  STATES 

Until  the  fall  of  1944,  radar  training  in  the  continental  United 
States  was  conducted  on  a  relatively  small-scale.  Instruction  in  radar 
navigation  and  radar  bombing  was  given  regularly  at  only  two  schools : 
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Langley  Field,  Va.,  and  Boca  Raton  Army  Air  Field,  Fla.  At  each  of 
these  schools,  graduate  bombardiers  and  navigators  were  given  4  weeks 
of  radar  observer  training.  The  combined  student  flow  was  approxi¬ 
mately  210  per  month.  At  that  time,  Langley  Field  was  an  overseas 
replacement,  trnining  center,  and  most  of  the  graduates  of  both  schools 
received  additional  training  while  awaiting  transportation  to  the 
European  Theater. 

A  third  radar  observer  school  was  organized  in  April  1944  by  the 
Second  Air  Force  to  train  radar  observers  for  B-29  operations  in  the 
Pacific  Theater.  This  school  was  located  at  Smoky  Hill  Army  Air 
Field,  Salina,  Kans.,  and  trained  classes  of  approximately  25  per 
month.  The  training  consisted  of  both  ground  and  aerial  instruction 
in  radar  navigation  and  bombing.  rl'he  curriculum  stressed  training 
in  off-set  bombing,  a  technique  for  bombing  an  obscured  target  by 
making  computations  with  reference  to  a  visible  radar  return  located 
a  known  distance  and  direction  from  the  target.  Training  continued 
until  August  1944,  when  the  school  was  disbanded  and  the  instruc¬ 
tional  staff  was  transferred  to  the  radar  observer  schools  at  Boca  Raton 
and  Langley  Field. 

In  the  fall  of  1944,  after  almost  a  year  of  increasing  success  with 
radar  navigation  and  bombing,  it  was  decided  to  expand  the  training 
program  in  the  United  States.  Because  of  a  relative  excess  of  pilot3 
and  shortage  of  bombardiers  and  navigators,  a  decision  was  made  to 
attempt  to  train  the  former  as  radar  observers.  The  training  course 
was  increased  to  16  weeks  and  included  instruction  in  non-radar  navi¬ 
gation  and  bombing.  After  about  1  month,  the  16-wcek  courso  was 
abandoned  because  the  motivation  of  pilots  for  such  training  was  low. 
The  selection  of  prospective  students  from  bombardiers  and  navigators 
was  resumed  and  the  duration  of  the  courso  was  fixed  at  10  weeks.  At 
the  time  of  entering  radar  observer  training,  bombardiers  wero  re¬ 
quired  to  have  had  navigation  training  in  advanced  bombardier  school 
and  navigators  wero  required  to  take  4  weeks  of  prerndar  training  in 
bombing. 

In  early  1945,  a  new  radar  observer  school  was  established  at  Victor¬ 
ville  Army  Air-Field,  Calif.  The  combined  student  flow  of  tho  three 
stations  was  soon  increased  to  500  students  per  month.  With  this 
rapid  growth  came  many  problems  of  curriculum  standardization. 
These  problems  arose  in  part  because  the  different  training  stations 
were  equipped  with  different  radar  sets.  Tho  AN/APS-15  and  tho 
AN/A  PS-1 5A  were  used  at  Langley  Field  while  the  AN/APQ-13  was 
used  at  Boca  Raton  and  at  Victorville.  However,  most  problems  of 
curriculum  standardization  arose  between  schools  using  the  same  set, 
as  to  which  operating  procedures  were  best  and  how  thej  should  bo 
taught.  The  question  of  which  techniques  were  most  appropriate  to 
tho  different  theaters  of  operation  was  particularly  troublesome. 
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The  demand  for  radar  observers  continued  to  increase.  In  l ho 
spring  of  1945,  a  fourth  radar  observer  school  was  activated  at 
Williams  Army  Air  Field,  Ariz.,  and  in  the  early  summer  of  1945.  a 
fifth  school  was  established  at  Yuma  Army  Air  Fit  Id,  Ariz.  The 
combined  monthly  flow  for  the  5  schools  was  1,000  students.  Very 
few  students  graduated  from  ’.cr  of  the  new  schools,  however, 
since,  with  the  end  of  the  war,  radar  training  was  immediately 
curtailed. 

SUMMARY 

This  chapter  gives  a  brief  description  of  the  training  of  the  radar 
observer  and  his  place  in  the  war-time  operations  of  the  Army  Air 
Force.  It  is  divided  into  three  sections.  The  fundamental  principle 
of  radar,  it  is  pointed  out  in  the  first  section,  is  similar  to  that  of  the 
echo.  Short  pulses  of  radio  energy  are  sent  out  and  a  receiving  unit 
presents  in  visual  form  the  direction  and  distance  of  the  object  which 
reflects  the  pulses.  1 

The  second  section  is  devoted  to  the  use  of  airborne  radar  in  combat 
operations.  This  account  begins  with  the  early  uses  of  ground  radar 
in  detecting  aircraft  and  the  installation  of  the  first  airborne  sets  in 
fighter  aircraft.  Search  sets  were  installed  in  aircraft  as  a  part  of 
the  anti-submarine  campaign;  these  sets  were  the  forerunners  to  tho 
sets  operated  by  the  radar  observer.  Tho  RAF  was  the  first  air  force 
to  use  radar  as  an  aid  to  navigation  and  bombing.  In  time  tho  Eighth 
Air  Forco  had  radar-equipped  pathfinder  aircraft  and  crews  operating 
on  many  of  its  bombing  missions.  A  school  was  also  set  up  by  this 
Air  Force  to  train  radar  observers.  Air  forces  operating  in  tho  Medi¬ 
terranean  and  Pacific  Theaters  made  increased  use  of  radar  aids  as 
the  war  continued. 

The  final  section  of  tho  chapter  outlines  tho  training  program  for 
radar  observers  set  up  in  tho  United  States.  As  combat  operations 
increased  it  was  necessary  to  expand  training  facilities  in  this  country. 
Within  a  period  of  months  tho  number  of  training  stations  was  in¬ 
creased  from  2  to  5  and  the  monthly  student  flow  from  250  to  1,000. 
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CHAPTER  THREE _ _ _ 

Survey  of  Research1 


The  purpose  of  this  chapter  is  to  review,  in  chronological  order, 
the  major  research  studies  accomplished  in  the  selection  and  training 
of  radar  observers.  The  presentation  is  in  the  form  of  a  survey; 
detailed  accounts  of  specific  research  accomplishments  may  bo  found 
in  subsequent  chapters. 

The  studies  to  be  mentioned  will  be  introduced  in  relation  to  the 
development  of  three  more  or  less  independent  research  organizations. 
The  three  organizations,  in  the  order  in  which  they  will  be  presented, 
are  first,  the  National  Defense  Research  Committee  Project  SC-70, 
NS-146,  referred  to  as  the  NDRC  Project;  second,  the  AAF  Aviation 
Psychology  Program,  Aircrew  Evaluation  and  Research  Detachment 
No.  1,  the  Eighth  Air  Force,  to  be  referred  to  as  the  AERD  No.  1,  and 
third,  the  AAF  Aviation  Psychology  Program,  Psychological  Re¬ 
search  Project  (Radar).  In  addition,  mention  will  be  made  of  the 
selection  of  radar  observers  by  Psychological  Research  Project  (Navi¬ 
gator)  and  Headquarters,  AAF  Training  Command. 

NATIONAL  DEFENSE  RESEARCH  COMMUTED— PROJECT 

SC-70,  NS-1-16 

The  broad  task  assigned  the  NDRC  Project  upon  its  initiation  in 
February  1913,  was  research  upon  psychological  problems  of  radar 
operation  in  both  the  Army  and  Navy.2  In  carrying  out  this  assign¬ 
ment  personnel  of  the  project  conducted  numerous  investigations, 
some  related  primarily  to  ground  radar  installations,  others  primarily 
to  sets  employed  by  naval  vessels  or  aircraft.  In  this  review,  only  re¬ 
search  bearing  upon  the  problems  of  airborne  radar  will  be  mentioned. 

Among  the  airborne  radar  assignments  undertaken  by  the  NDRC 
Project,  one  of  the  first  was  a  job  analysis  of  the  operation  of  equip¬ 
ment  designated  as  ASV  (air-to-surface-vcssei)  a  typo  of  airborno 

‘Written  by  Sjjt.  Albert  II.  Hastorf. 

‘Applied  I’uyeholoKy  Panel,  NDUC,  Final  report  In  nummary  of  work  on  the  selection  and 
training  of  radar  oporntorB,  Research  Report  No.  19,  2-1  September  10. |5.  The  following 
personnel  participated  In  the  Project;  Donald  15.  LlndMcy,  director,  Irving  11.  Anderson, 
Alfred  I,.  Baldwin,  Charles  II.  Ilrldgman,  Robert  H.  Daniel,  John  0.  Dnrley,  Hubert  It. 
Orcher,  Edward  1*.  ilornc,  Edward  A.  Jerome,  William  II.  Llehto,  Thomas  E.  McCulloch, 
Fred  McKinney,  Karl  U.  Smith,  0.  Raymond  Stone,  Edward  J.  Sweeney,  Onrth  J.  Thomas. 


11 


vailin'  S'*;)  ‘•vavch  equipment.  ASV  used  cither  (he  SCR  717. V  nr  SCR 
717B  sets  which,  like  the  sets  i..-:ed  hy  the  radar  observer,  have  a  PPl 
scope  and  consequently  require  similar  operational  skills  and  abilities. 
Following  this  analysis  personnel  of  the  project  prepared  In  selection 
tests  dealing  principally  w  ith  perceptual  aspects  of  the  operator’s  task. 
These  tests  were  believed  to  measure  speed  and  accuraej  of  percep¬ 
tual  discrimination,  alertness,  persistence,  and  the  ability  to  make 
quick  judgments.  To  establish  time  limits  and  scoring  method:-,  the 
tests  were  administered  experimentally  at  Camp  Murphy,  Fla.;  Boca 
Ration  Field,  Fla;  Langley  Field,  Vn.;  and  at  S  stations  under  the 
AAF  Tactical  Center,  Orlando,  Fla. 

Several  tests  from  this  group  of  15  were  administered  by  A  ERL) 
No.  1  as  part  of  a  selection  research  project  to  bo  described  below.3 
Those  found  by  AERD  No.  1  to  be  most  predictive  of  success  in  radar 
observer  training  were  included  later  in  a  test  battery  administered 
to  potential  radar  observer  students  in  the  United  States  by  testing 
teams  from  Headquarters,  AAF  Training  Command.  Other  tests 
of  this  group  "wore  used  by  Psychological  Research  Project  (Naviga¬ 
tor)  during  the  period  when  that  organization  was  responsible  for 
radar  observer  student  selection.  Psychological  Research  Project 
(Radar)  prepared  machine  scoreablo  forms  of  others  for  inclusion 
in  a  battery  of  experimental  selection  tests. 

The  NDRC  Project  also  contributed  to  radar  observer  research  in 
the  areas  of  proficiency  measurement  and  of  training  methods.  Two 
comprehensive  printed  proficiency  tests  were  coustnicted,  one  for  the 
AN/APS-15  set,  and  another  for  the  AN/APQ-7  set.  A  film  trainer 
was  developed  which  used  motion  pictures  of  scope  returns  as  briefing 
and  reconnaissance  aids.  A  manual  was  prepared  to  aid  students  in 
training  as  radar  observers  for  low7  altitude  radar  bombing.  A  photo 
bomb  scoring  computer  was  constructed.  One  of  the  project’s  most 
exlensivo  studies  was  an  extended  training  experiment  designed  to 
ascertain  the  effect  on  proficiency  of  continued  training  beyond  the 
normal  duration  of  the  AAF  radar  observer  training  course. 

AIRCREW  EVALUATION  AND  RESEARCH  DETACHMENT 

NO.  I4 

AERD  No.  1  consisted  of  G  officers  and  15  enlisted  men  of  the  AAF 
Aviation  Psychology  Program  who  were  detached  from  May  to 
August  1911,  and  assigned  to  duty  with  the  Eighth  Air  Force  in  Eng¬ 
land.  At  this  stage  in  the  development  of  the  air  war  the  increasing 
use  o *  radar  as  an  aid  to  nevigation  and  bombing  had  emphasized  the 
importance  of  choosing  well-qualified  personnel  for  training  as  radar 

*  Topley,  W.  M.,  c<f.,  Pei/riiolo'jtcul  rcscurch  tn  the  Ihrntcr*  oj  tcur.  AAF  aviation  psychol¬ 
ogy  p.'ogrnm  research  reports,  No.  17.  Washington,  Government  minting  Office,  1047.  . 

*  im. 
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observers.  As  a  result,  AERD  Xo.  1  a-signed  G  of  its  members  to 
specialized  research  in  the  selection  of  radar  observer  students.5  A 
job  analysis  was  made  of  the  particular  skills  and  abilities  required 
by  the  airborne  radar  observer  in. training  at  the  Eighth  Air  Forco 
Radar  Observer  School.  Following  this  analysis  '20  printed  selection 
tests  were  chosen  for  validation  against  course  grades  at  the  school. 
The  tests  validated  include  several  of  those  developed  by  tho  NDRC 
Project,  several  originated  by  AERD  No.  1,  and  a  large  group  from 
the  AAF  Air-crew  Classification  Battery.  Also  validated  were 
stanincs  for  bombardier,  navigator  and  pilot.  Of  the  20  tests,  the  1 
most  valid  were  later  to  constitute  the  radar  observer  selection  battery. 

Another  major  undertaking  of  AERD  No.  1  was  tho  construction 
of  a  comprehensive  printed  proficiency  test  for  radar  observers.  In¬ 
cluded  in  this  test  was  a  section  in  which  the  student’s  task  was  to 
navigate  through  a  simulated  radar  mission.  Navigation  was  carried 
out  with  the  aid  of  a  full-size  plotting  chart  of  northwestern  Germany 
and  was  dependent  upon  the  correct  interpretation  of  a  series  of 
photographs  of  tho  radar  scope. 

SELECTION  OF  RADAR  OBSERVERS 

As  the  importance  of  the  radar  observer  in  the  European  Theater 
increased,  the  desirability  of  screening  potential  students  in  the  United 
States  became  apparent.  In  July  1914,  the  Psychological  Research 
Project  (Navigator),  at  the  direction  of  Headquarters,  AAF,  as¬ 
sembled  a  selection  battery  to  be  administered  to  potential  radar 
students  at  tho  advanced  navigation  schools.  Three  tests  developed 
by  tho  NDRC  Project  and  a  preference  blank  were  chosen.  The  bat¬ 
tery  was  administered  to  advanced  students  who  either  had  navigator 
stanines  of  eight  and  above  or  ranked  in  the  upper  third  of  their  class. 
Tho  first  administration  took  place  in  July  1914,  at  Hondo  Army  Air 
Field,  Tex.;  later  administrations  were  carried  out  at  three  other 
navigation  schools.  Routine  testing  for  screening  purposes  was  con¬ 
ducted  by  personnel  of  these  schools  and  continued  until  November 
1944. 

On  9  November  1944,  Headquarters  AAF  Training  Command  estab¬ 
lished  airborne  radar  observer  selection  teams  for  the  purposo  of 
administering  a  selection  battery  based  upon  the  validation  studies 
of  AERD  No.  1.  In  May  1945,  the  Navigation  Proficiency  Test,  de¬ 
scribed  in  chapter  5,  was  added  to  this  battery  for  selecting  students  in 
advanced  bombardier  training.  Tho  airborne  teams  continued  selcc- 
tion  testing  until  July  1945. 

*  Major  n,  von  It.  Gilmer,  Captain  Stuart  W.  Cook,  Lt.  William  M.  Wheeler,  T/8gt. 
UtiSHcll  W.  lJornetneler,  T/Sgt.  Robert  B.  Miller,  Sgt.  Philip  II.  Krcldt. 
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psychologic  w.  keskarcii  project  (uadah) 

In  the  fall  of  19!  J.  afl'-r  a  y-  ir  of  increa -cd  suvo-.:.  in  -t r:i t  -yric 
bombing  v.ith  indur.  the  AAF  <ml:ir':  1  upon  a  gix  uly  expanded 
program  for  the  training  of  radar  ob-i-.-vi-r-.  T'.-*  I  raining  program, 

until  that  time,  had  been  com-w.trs’.ed  at  2  training  nations.  Plans 
were  made  to  inrrea-e  the  mnnher  of  .-tat ions  to  a.  and  later  to  a.  and 
tocxpuml  the  total  -Indent  flow  from  250  to  1  .<  "'0  mb  m-  ]  -er  month.6 
These  developments  made  desirable  the  e-tabb-hmeni  of  an  AAF 
psychological  project  for  specialized  radar  observer  research. 

Activation  of  the  Project 

Early  in  September  1014.  Lt.,  Col.  A.  Paul  Horst,  Commandin'! 
Oflicer  of  AERD  No.  1,  tteconipaniod  by  Map  Beverley  von  II.  Gilmer 
and  Capt.  Stuart  W.  Cook,  reported  to  the  Air  Surgeon,  Washington, 
to  discuss  the  preliminary  findings  and  plans  for  completion  of  the 
work  undertaken  by  that  detachment  while  with  the  Eighth  Air 
Force.  At  this  time  the  first  conferences  were  held  relative  to  the 
need  for  a  psychological  project  in  the  selection  and  training  of  radar 
observers.  Later  this  need  was  discim-ed  with  the  Surgeon  at  Head¬ 
quarters,  AAF  Training  Command,  Fort  Worth,  Tex.,  and  with  Col. 
William  M.  Garland,  then  deputy  for  training  and  operations  at 
Langley  Field,  Va.  It  was  decided  at  these  discussions  to  activate  a 
project  under  the  direction  of  Captain  Cook  and,  because  of  the 
urgency  of  the  work,  to  assign  personnel  to  Langley  Field  on  tem¬ 
porary  duty  prior  to  official  activation. 

Arrangements  were  made,  also,  to  enlist  the  assistance  of  established 
psychological  units  and  projects.  Two  officers  from  the  Psychology 
Department,  School  of  Aviation  Medicine,  Randolph  Field,  Tex.,  wTere 
assigned  on  temporary  duty  to  assist  in  the  planning  for  experimenta¬ 
tion  with  apparatus  tests.  The  Director  of  Psychological  Research 
Project  (Bombardier)  assisted  with  plans  for  proficiency  measure¬ 
ment.  An  ollicer  from  the  Psychological  Research  Unit,  San  Antonio, 
collaborated  in  tbe  preparation  of  plans  for  experimentation  with 
printed  relection  tests.  By  October  1914,  a  group  of  9  officers  and 
15  enlisted  nieu  had  been  assembled  at  Langley  Field.  Working 
quarters  were  established  for  this  group  on  the  flight  lino  where  other 
flying  and  training  activities  were  concentrated. 

On  1  I)  cci  inbcr  1914,  the  Psychological  Research  Project  (Radar) 
was  officially  activated  by  Headquarters,  AAF  Training  Command.7 

•  The  find  tv.-o  nrlioola  were  nit  unfed  at  I.nnglcy  Field  nnd  at  Boca  Itnton  AAF.  Tbe 
former  wan  rcNiiorndble  for  AN/Al’S-15  and  AN/APS-15A  training  wbllo  the  lalter  laught 
AN/Al’Q-13  nntl  AN/Al’Q-7.  A  llilnl  lichool,  Victorville  AAF  wns  OHluldlHhed  for 
AN/Al’Q-13  training.  It  win  Mipidemi-nted  Inter  by  Yuma  AAF  and  Williams  Field.  Tbe 
latter  undertook  nil  AN/Al’Q-7  training. 

’  Letter,  HendiuinrterH  AAF  Training  Comninnd,  to  Commanding  Oenernl,  AAF  Enstcrn 
Technical  Training  Cornnmnd,  Subject  :  l-bstnlillKliment  of  Psychological  lteaenrch  Project 
(llndnr),  25  November  1 0-t-l,  File  353  lUdnr. 
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Tho  majority  of  personnel  t-t  allotted  at  Langley  Fiolil  on  temporary 
duty  wore  then  permanently  assigned. 

Research  Object  Ives  and  Priorities 

The  letter  of  activation  states  the  mission  of  the  project  as  follows: 
a.  The  development  of  aptitude  tests  for  the  selection  of  radar  op¬ 
erators.  b.  The  development  of  radar  operator  proficiency  criteria 
against  which  to  validate  aptitude  tests.  <\  The  investigation  of  con¬ 
ditions  of  optimally  ellicient  use  of  trainers  and  training  methods. 
</.  Conduct  of  research  studies  on  other  psycholog  val  problems  to  bo 
directed  by  this  headquarters.8 

Priorities  were  assigned  to  research  objectives  on  the  basis  of 
practical  circumstances,  under  which  the  project  began  its  work.  First, 
and  highest  priority  was  given  to  the  development  of  proficiency 
tests  and  checks.  Second  priority  was  assigned  to  the  validation  of 
selection  tests.  Third  priority  was  placed  upon  instructor  selection 
and  evaluation.  Lowest  priority  was  given  to  the  investigation  of 
trainers  and  training  methods. 

Several  important  considerations  made  tho  development  of  pro¬ 
ficiency  measures  most  urgent.  Tho  training  program  was  now  and 
there  was  an  acutely  felt  need  for  acceptable  methods  of  evaluating 
students.  Supervisory  training  personnel  under  the  leadership  of 
Colonel  Garland,  were  dissatisfied  with  the  methods  Hint  had  been 
hurriedly  improvised  and  were  receptive,  consequently,  to  proposals 
regarding  new  types  of  proficiency  measurement.  In  addition,  it 
appeared  probable  that,  until  ncccptablo  proficiency  measures  were 
constructed,  there  would  bo  available  no  adequate  criterion  against 
which  to  validate  selection  tests.  Of  tho  possiblo  criteria,  one,  tlio 
pass-fail  criterion,  was  eliminated  because  tho  demand  by  the  oper¬ 
ational  air  forces  for  radar  observers  did  not  nllow  the  failuro  of  in¬ 
ferior  students.  Another,  instructor  grades,  appeared  likely  to  be  of 
doubtful  value  because  the  rapid  expansion  of  training  necessitated  tho 
use  of  many  instructors  with  no  previous  teaching  experience  nnd 
others  with  little  or  no  motivation  to  teach.  A  third,  bombing  accu¬ 
racy,  was  made  impractical  by  tho  lack  of  sufficient  photographic 
equipment  at  the  radar  training  stations. 

Work  on  the  validation  of  selection  tests,  assigned  second  priority 
because  of  the  necessity  for  immediate  development  of  proficiency 
measures,  was  only  slightly  delayed  by  this  emphasis.  While  it  was 
not  possiblo  to  begin  the  development  of  new  selection  tests,  early 
attention  was  given  to  tho  assembly  nnd  administration  of  n  battery 
of  selection  tests  available  from  other  sources.  It  was  expected  that 
development  of  new  tests  would  not  bo  long  delayed  since  plans  called 
for  a  standard  training  program  in  which  identical  proficiency 

*  paragraph  2. 
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measure's  would  be  used  at  all  training  stations.  The  fact  that  such 
standardization  was  not  achieved  multiplied  greatly  the  work  in 
proficiency  measmement  and  unduly  delayed  selection  test  develop¬ 
ment. 

Research  on  instructor  selection  and  evaluation,  while  assigned 
third  priority  in  the  project’s  emphasis,  was  greatly  needed  and  re¬ 
quired  a  relatively  small  investment  of  personnel.  Validation  of  the 
instructor  selection  tests  was  never  accomplished  because  of  tbc  delay 
in  accumulating  suitable  criteria  of  instructor  proficiency.  Lowest 
priority  was  given  to  research  projects  in  training.  While  significant 
training  problems  were  present,  the  project  followed  the  policy  of  the 
Aviation  Psychology  Program  in  emphasizing  research  in  personnel 
selection  and  evaluation. 

Survey  of  Project  Research 

One  of  the  first  tasks  of  the  project  was  to  become  adequately  ori¬ 
ented  to  the  radar  observer’s  tasks  and  to  the  technical  details  of 
various  types  of  radar  equipment.  Lectures,  demonstrations,  obser¬ 
vations  and  orientation  flights  were  specially  arranged  for  this  pur- 
post.  Members  of  the  project  were  successively  enrolled  in  the  course 
at  Langley  Field  throughout  the  duration  of  the  training  program. 
In  this  course  project  personnel  participated  in  all  ground  and  flying 
training  us  regular  students,  and  accumulated  sufficient  numbers  of 
hours  of  flying  time  to  solo  as  radar  observers. 

The  first  2  months  of  the  project’s  activity  were  directed  primarily 
toward  the  completion  of  a  battery  of  standardized  proficiency 
measures  for  the  various  types  of  radar  equipment  used.  Early  in 
January  1915,  a  battery  of  three  proficiency  tests  and  three  perform¬ 
ance  checks,  applicable  to  botli  AN/APS-15  and  AN/APQ-13  equip¬ 
ment,  was  presented  at  the  conference  of  the  Radar  Standardization 
and  Advisory  Board.  Following  the  presentation,  the  board  adopted 
a  recommendation  “that  the  phase  check's  and  examinations  which  are 
set  up  by  the  Psychological  Rcscnrch  Project  (Radar)  be  regarded 
ns  the  only  examinations  to  be  given  and  that  no  other  special  exam¬ 
inations  be  administered.”  Informal  invitations  were  issued  at  that 
time  by  the  Deputies  for  Training  and  Operations  at  all  radar  train¬ 
ing  stations  to  install  proficiency  measures  at  the  earliest  opportunity. 

^During  January,  the  initial  battery  of  proficiency  tests  and  per¬ 
formance  checks  was  further  developed  and  refined.  This  work  en¬ 
tailed  conferences  with  training  personnel  at  Langley  Field,  participa¬ 
tion.  in  training  flights,  and  research  with  the  AN/APQ-13  set  lit  the 
Boca  Raton  radar  observer  school.  Detailed  memoranda  were  pre¬ 
pared  dealing  with  procedures  for  administering,  scoring,  and  safe¬ 
guarding  tests  and  performance  checks.  An  experimental  battery  of 
six  apparatus  selection  tests  was  installed  at  Langley  Field  under  the 
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supervision  representative  of  tho  Department  of  Psychology, 

tkhool  of  A  ition  Medicine.  Plans  were  formulated  for  tho  first 
experimental  battery  of  printed  selection  tests  and  for  tho  validation 
of  tests  previously  administered  at  air-crew  classification  centers  and 
by  the  airborne  radar  observer  selection  teams. 

Early  in  February  1915,  a  new  phase  of  the  project’s  work  began. 
Administration  of  a  battery  of  experimental  selection  tests  was 
initiated  at  the  three  radar  training  stations  then  in  operation  and 
testing  with  tho  apparatus  test  battery  began  at  Langcly  Field.® 
Simultaneously,  the  introduction  of  standardized  proficiency  tests  and 
performance  cheeks  at  the  three  radar  observer  schools  was  attempted. 
At  two  of  tho  schools,  the  installation  proceeded  essentially  according 
to  plan  and  with  considerable  success.  At  the  third,  the  battery  was 
inapplicable  because  of  unanticipated  differences  in  curriculum  and 
in  operating  procedure.  Many  similar  incidents  to  follow  introduced 
the  project  to  problems  which  were  never  satisfactorily  solved.  Soon, 
for  example  tho  nature  of  aerial  training  at  another  of  the  schools  was 
radically  changed.  The  activation  of  additional  radar  training  sta¬ 
tions  with  new  local  variations  in  training  techniques  and  proficiency 
requirements  presented  still  other  complications. 

To  theso  curricular  and  administrative  problems  were  added  ques¬ 
tions  concerning  tho  manner  in  which  performance  checks  were  be¬ 
ing  administered.  It  soon  became  evident  that  checks  administered  by 
instructors  could  not  be  standardized  and  that  a  small  group  of  trained 
and  specialized  examiners  was  necessary. 

In  the  period  between  tho  development  of  these  difficulties  and  tho 
end  of  the  war,  tho  project’s  work  involved  concurrent  attacks  on  all 
phases  of  its  research  assignment.  The  difficult  task  was  undertaken 
of  preparing,  revising,  and  duplicating  the  extensive  proficiency  bat¬ 
tery  to  fit  each  more  or  less  unique  local  curriculum  and  operating  pro¬ 
cedure.  Considerable  effort  by  project  personnel  resulted  in  the  estab¬ 
lishment  of  specialized  examiners  at  all  training  stations.  A  job 
description  and  job  analysis  were  completed,  plans  for  a  second  battery 
of  experimental  selection  tests  were  formulated,  administration  of  ex¬ 
perimental  selection  tests  was  continued,  an  instructor  evaluation  pro¬ 
gram  was  installed,  and  refinements  were  made  in  the  methodology  of 
constructing  proficiency  tests  and  performance  checks. 

Following  Y-J  Day  existing  research  projects  were  terminated  and 
project  personnel  on  temporary  duty  at  the  various  radar  training 

•Project  personnel  responsible  nt  different  tlmco  for  tho  administration  of  thcce  testa  r.t 
tho  rccpcctlvo  rotations  were:  for  I.analey  Field:  Srrt.  Hyman  Heller;  Cpl.  Jnmen  It.  Holt; 
Cpl.  James  C.  McClure;  CpL  ltobcrt  J.  Pnttcrnon;  Ffc.  Cordon  L.  Fuller,  Jr.;  Pvt.  Donah! 
C.  nennett;  and  Pvt.  Martin  S.  Mnltenfort ;  for  Boca  Itaton  AAF;  Bst.  Norman  Graff;  Cpl. 
Nelson  It.  Nall ;  Pfc.  Jack  B.  McCollorn  ;  and  Pfc.  Gordon  B.  Puller,  Jr. ;  for  Victorville  AAF: 
S,7t.  Michael  Green  ;  Cpl.  Wilbert  H.  Schwotzer  ;  for  Ynmn  AAF:  Cpl.  Wilbert  II.  Bchwotser ; 
and  for  Williams  Field  ;  Sst.  Gerald  S.  Blum  and  Cpl.  Douglao  W,  Bray. 


stations  returned  to  Langley  Field.  The  project  then  directed  all 
activities  towards  completing  the  task  of  analyzing  the  data  which 
had  been  obtained. 

Personnel 

In  (he  following  roster  arc  listed  personnel  permanently  assigned 
to  the  project.  The  names  of  persons  associated  with  tho  project 
throughout  the  major  portion  of  its  research  history  arc  preceded  by 
an  asterisk  (*). 


Officer  Personnel 

*Capt.  Stuart  W.  Cook  ( Director )  •Lc.  Lewis  G.  Carpenter,  Jr. 

Capt.  Ike  II.  Harrison  *Lt.  George  S.  Klein 

Capt.  William  F.  Long  *Lt.  Stuart  Loftier 

Capt.  Gabriel  P.  Ofiesh  Lt.  Sol.  AT.  Roshal. 

•Capt,  II.  Richard  Van  Saun  ( Assistant  Director ) 

Enlisted  Personnel 


"Technical  Sgt.  Geo.  N.  Bollinger 
Technical  Sgt.  Sanford  J.  Mock 
Technical  Sgt.  Hyman  Schmiorcr 
Stall  Sgt.  Trent  E.  Bess  nt 
•Staff  Sgt.  Lester  I.  Foster 
Staff  Sgt.  Roland  E.  Johnston, 
Jr. 

Staff  Sgt.  Harold  F.  Kunsman 
•Staff  Sgt.  Richard  T.  Mitchell 
"■Staff  Sgt.  Bernard  C.  Sullivan 
Stalf  Sgt.  William  J.  Woywod 
"■Sgt.  Alfred  S.  Arnott 
•Sgt.  Gerald  S.  Blum 
•Sgt.  Stanley  Blumbcrg 
-'‘Sgt.  Nathaniel  L.  Gage 
"Sgt.  Norman  Graff 
•Sgt.  Michael  Green 
•Sgt.  John  S.  Harding10 
"'Sgt.  Albert  H.  Hastorf 
•Sgt.  Hyman  Heller 
Sgt.  Ted  P.  Kiscii'as 
•Sgt.  Philip  II.  Kriedt 
•Sgt.  William  J.  Mungan 
*Sgt.  Samuel  D.  Morford 
*Sgt„  Sheldon  II.  Nerby 


Sgt.  Albert  Pepitone 
Sgt,  Harold  L.  Raush 
*Sgt.  Harold  I.  Roth 
Cpl.  Alfred  D.  Antilla 
•Cpl.  Arlene  E.  Babcock 
*Cpl.  Douglas  W.  Bray 
Cpl.  Irving  Fudeinan 
*Cpl.  John  V.  Hcnnessy 
•Cpl.  James  R.  Holt 
•Cpl.  Harold  H.  Kelley 
*Cpl.  Robert  H.  Koch 
Cpl.  Alary  E.  Loomis 
•Cpl.  John  F.  AlacNaughton10 
*Cpl.  James  C.  McClure,  Jr. 
Cpl.  Owen  R.  Alunger 
Cpl.  Nelson  R.  Nail 
•Cpl.  Robert  J.  Patterson 
•Cpl.  Wilbert  II.  Schwotzer 
Cpl.  Vernon  S.  Scott 
Cpl.  James  E.  Skowronsld 
*Cpl.  Ilyman  Sofer 
Ffc.  James  II.  Anderson 
*Pfc.  Donald  G.  Bennett 
Pfc.  Robert  J.  Blount 
Pfc.  Dwano  R.  Collins 10 


,e  CoimolHHloncil  na  2il  lieutenant,  A.  O.  D.  on  leaving  the  project. 
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Pfc.  Billy  S.  E'liot 
Pfc.  Sanford  Goldstone 
l’fc.  Edmund  W.  Lyons 
* Pfc.  Jack  L.  McCollom 
Pfc.  Kenneth  M.  Mitchell 


Pfc.  Thomas  B.  Morgan 
Pfc.  Roger  L.  Murrol 
Pfc.  Gordon  L.  Puller,  Jr. 
Pfc.  Hyman  Rogosin 
*Pvt.  Martin  S.  Maltenfort 


SUMMARY 


This  chapter  has  reviewed  the  research  accomplished  by  this  and 
other  organizations  in  the  selection  and  training  of  radar  observers. 
The  purpose  of  the  survey  has  been  to  present  n  chronology  of  re¬ 
search  activities  to  serve  as  a  framework  for  the  more  detailed  descrip¬ 
tions  of  the  research  accomplishments  found  in  the  subsequent 
chapters. 

The  survey  is  organized  primarily  around  the  development  of  three 
more  or  less  independent  research  organizations.  It  begins  with  the 
work  of  Propeet  SC-70,  NS-liG,  of  the  National  Defense  Research 
Committee  in  February  of  1013.  Although  this  project  performed 
research  on  both  air-borne  and  ground  radar  for  the  Army  and  the 
Navy,  this  chapter  reviews  only  its  research  on  the  selection  and  train¬ 
ing  of  air-borne  operators.  Following  a  job  analysis  of  the  operation 
of  the  air-borne  radar  equipment  used  in  sea  search,  the  project  devel¬ 
oped  15  selection  tests  dealing  principally  with  the  perceptual  aspects 
of  the  operator’s  task.  Among  its  contributions  to  research  in  the 
areas  of  proficiency  measurement  and  training  methods,  were  tho  con¬ 
struction  of  two  comprehensive  printed  proficiency  tests  for  radar 
observers  and  the  conduct  of  an  experiment  on  the  effect  of  extended 
training. 

Tho  Air-crew  Evaluation  and  Research  Detachment  No.  1,  the  second 
organization  discussed,  conducted  its  research  at  a  radar  observer 
school  in  the  Eighth  Air  Force.  Here  a  job  analysis  was  made,  and 
20  printed  selection  tests  wero  validated  against  course  grades.  ’The 
four  most  valid  of  these  tests  were  later  to  constitute  the  radar  observer 
selection  battery,  administeied  to  all  prospective  radar  observer  stu¬ 
dents  by  testing  teams  under  tho  direction  of  Headquarters  AAF 
Training  Command.  Prior  to  tho  use  of  this  battery,  Psychological 
Research  Project  (Navigator)  had  administered  3  NDRC  Project 
selection  tests  to  prospective  students. 

In  the  fall  of  1914  tho  Psychological  Research  Project  (Radar) 
was  activated.  Its  mission  was  to  perform  research  on  the  psychologi¬ 
cal  problems  encountered  in  the  training  of  radar  observers  in  the 
AAF  Training  Command.  Early  emphasis  was  placed  by  the  project 
on  tho  development  of  a  comprehensive  battery  of  proficiency  tests 
and  checks.  Although  school  differences  in  curriculum  and  operating 
procedure  required  numerous  revisions  of  these  measures,  they  were 
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eventually  placed  in  general  use  at  all  training  stations.  The  project 
conducted  a  selection  research  study  in  which  all  entering  radar 
observer  students  were  administered  a  lengthy  battery  of  experimental 
printed  tests.  In  addition,  experimental  apparatus  tests  were  admin¬ 
istered  at  one  school.  These  tests  along  with  scores  from  the  air  crew 
classification  and  radar  observer  selection  batteries  were  validated 
against  course  grades. 
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Job  Description  and  Analysis1 


GENERAL  CONSIDERATIONS 

This  chapter  describes  and  analyzes  the  radnr  observer’s  job  for 
the  purposes  of  psychological  research.  It  includes  the  job  in  train¬ 
ing,  the  job  in  combat,  and  indications  of  probable  future  trends. 

Reason  for  Describing  a  Job 

Some  description  of  a  job  forms  the  basis  of  nil  psychological  re¬ 
search  upon  thnt  job.  More  often  than  not  this  important  first  step 
has  been  carried  out  informally  or  not  nt  all.  Nevertheless,  some 
definition  of  the  job  has  been  the  foundation  of  every  job  research 
program  even  if  it  was  only  implied  in  the  thinking  of  the  research 
psychologist. 

A  systematic  job  definition  can  serve  several  highly  important  func¬ 
tions.  First,  it  can  describe  the  specific  abilities  or  aptitudes  required 
by  the  job.  Aided  by  this  description,  psychologists  may  choose  or 
construct  selection  tests  which  appear  most  likely  to  measure  these 
particular  abilities  and  aptitudes.  Second,  a  job  definition  can  de¬ 
scribe  various  levels  of  proficiency  on  the  job.  Such  descriptions  are 
useful  in  developing  measures  of  proficiency.  Third,  a  job  definition 
can  provide  a  summary  of  the  tasks  and  skills  which  comprise  the 
job  for  the  purpose  of  setting  up  the  most  efficient  program  of  training 
personnel  for  the  job.  Fourth,  a  job  definition  can  describe  the  actual 
techniques  of  doing  the  job  and,  where  equipment  is  involved,  show 
how  the  equipment  is  actually  being  used.  This  information  often 
yields  clues  for  improving  techniques  of  using  existing  equipment, 
improving  the  equipment  itself,  or  better  adapting  it  to  the  abilities 
and  characteristics  of  the  average  individual  on  the  job. 

Methods  of  Describing  n  Job 

A  job  can  be  described  using  concepts  at  various  levels  of  analysis. 
At  one  extreme  is  a  description  of  what  the  individual  does  on  the 
job  in  simple,  nontechnical  language.  Such  as  description  can  bo 
written  by  a  person  unfamiliar  with  psychological  concepts.  This 
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type  of  description,  to  be  referred  to  hereafter  as  a  “job  description,” 
enij  loys  terms  which  arc  for  the  most  part  specific  to  the  job  and  not 
transferable  from  one  job  to  another.  At  the  other  extreme  is  the 
type  of  de-cription  which  will  be  referred  to  here  as  a  “job  analysis.” 
A  job  analysis  is  a  description  of  the  job  in  terms  of  psychological 
functions  or  factors.  These  functions  or  factors  represent  abilities 
and  skills  which  are  found  introspectively  6r  statistically  to  be  com¬ 
mon  to  many  jobs. 

Whether  a  job  description  or  job  analysis  is  to  be  preferred  in  a  par¬ 
ticular  research  program  depends  upon  which  of  the  four  purposes 
mentioned  above  is  paramount  to  the  job  research.  Job  proficiency  is 
most  validly  measured  in  terms  of  the  specific  tasks,  skills,  and  con¬ 
tents  of  the  job.  Navigation  proficiency,  for  example,  is  measured 
more  validly  by  navigation  problems  than  by  general  mathematics 
problems.  Consequen* ly,  a  job  description  provides  the  most  ade¬ 
quate  foundation  for  construction  of  proficiency  measures.  Training 
research  is  also  based  preferably  upon  a  job  description.  Only  if  a 
job  analyst  believes  in  a  great  deal  of  transfer  of  training,  will  lie 
base  a  training  program  upon  a  factor  definition  of  the  job  and  train 
individuals  in  general  functions  rather  than  in  specific  tasks. 

In  addition,  a  job  description  is  more  useful  than  a  job  analysis  in 
furnishing  the  basis  for  research  into  techniques  and  equipment. 
However,  even  in  such  research,  the  job  analysis  is  not  completely  with¬ 
out  use.  It  is  probable  that  an  identification  of  pertinent  factors, 
coupled  with  information  as  to  the  relative  levels  of  ability  in  these 
factors  possessed  by  the  available  population,  could  simplify  the  prob¬ 
lem  of  adapting  equipment  to  the  abilities  of  the  job  performers. 

A  job  description,  job  analysis,  or  definition  at  any  level  of  gener¬ 
ality  between  these  extremes  can  be  used  as  the  basis  for  selection  test 
research.  The  choice  depends  upon  the  extent  to  which  factor  theory 
is  accepted.  Factor  theory  rests  in  part  on  the  hypothesis  that  per¬ 
formance  on  most  of  the  tasks  in  contemporary  technology  can  be 
explained  by  a  limited  number  of  independent  functions  or  factors 
taken  in  various  combinations  and  amounts.  If  this  position  is  taken, 
a  job  analysis  is  made  of  tho  job  and  research  proceeds  with  already 
available  or  newly  constructed  factor  tests.  Such  tests  usually  have 
little  face  validity,  that  is,  they  have  little  specific  content  in  common 
with  tho  job.  The  basis  of  their  construction  is  that  they  require  tasks 
ami  operations  which  seem  to  have  something  fundamentally  in  com¬ 
mon  with  tasks  carried  out  on  the  job.  In  other  words,  they  require 
use  of  one  of  the  same  psychological  functions. 

It  eject  ion  of  factor  theory  leads  to  writing  a  job  description  and  the 
development  of  job  analogy  tests.  Such  tests  arc  construed  so  as  to 
roemblo  the  job  as  closely  as  possible.  In  actual  practice,  of  course, 
most  lest  construction  falls  somewhere  between  these  two  extremes. 
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The  advantages  and  disadvant ages  of  these  two  approaches  to  selec¬ 
tion  test  research  arc  as  follows: 

First,  factor  tests,  taken  singly,  tend  to  have  low  validity  while  job 
analogy  tests  have,  relatively  high  validity. 

Second,  factor  tests  have  low  intercorrelations,  while  job  analogy 
tests  have  high  intercorrelations.  Consequently,  a  combination  of  a 
number  of  factor  tests  may  yield  a  composite  score  of  useful  validity 
even  though  the  individual  tests  have  a  low  validity.  On  the  other 
hand,  the  combining  of  a  number  of  job  analogy  tests  adds  little  to  tho 
accuracy  of  prediction  obtained  by  any  one  test. 

A  third  consideration  is  that  a  number  of  factor  tests  have  been 
constructed  and  refined  and  arc  immediately  available  for  use  in  se¬ 
lecting  trainees  for  jobs  which  require  the  corresponding  abilities. 
Job  analogy  tests  have  to  be  specially  constructed  for  every  unique  job. 
On  the  other  hand,  the  factor  approach  is  limited  by  the  feet  that  for 
some  yeai’S  the  number  of  factor  tests  available  will  bo  inadequate  to 
the  requirements  of  complex  jobs.  It  is  anticipated  by  factor  theorists 
that  eventually  tests  will  he  constructed  to  measure  all  the  important 
factors,  thus  making  it  possible  to  set  up  a  selection  battery  for  any 
job  by  merely  assembling  the  pertinent  tests  as  indicated  by  a  job 
analysis. 

Fourth,  part  of  the  factor  content  of  jobs  which  require  operation  of 
complex  equipment  can  he  measured  with  inexpensive  printed  tests. 
On  the  other  hand,  job  analogy  tests  for  such  jobs  require  construction 
and  maintenance  of  expensive  testing  equipment. 

A  final  consideration  is  the  theoretical  objection  to  breaking  a  job 
into  elements  (factors)  for  the  purpose  of  selection  test  construction. 
The  contention  is  that  measuring  elementary,  independent  functions 
and  additively  combining  the  several  scores  yields  something  quite 
different  than  does  measuring  skill  on  the  total  task. 

One  method  of  evaluating  this  theory  consists  of  n  comparison  of 
job  analogy  tost  validity  with  the  composite  validity  of  factor  tests. 
If  the  theory  is  correct,  a  test  adequately  sampling  the  job  should  yield 
significantly  higher  validity  than  that  obtained  from  the  pertinent 
factor  tests.  Such  an  evaluation  can  be  invalid  either  because  tho  job 
analogy  test  does  not  adequately  sample  the  job  or  because  some  of  tho 
factors  involved  in  the  job  are  not  yet  known. 

Another  method  of  evaluation  consists  of  predicting  the  validities  of 
job  analogy  tests  (which  are  factorially  complex)  from  the  validities 
of  the  factors  they  have  been  found  to  contain.  These  predicted  valid¬ 
ities  should  fall  short  of  the  obtained  validities  if  the  job  analogy 
tests  measure  something  more  than  merely  the  sum  of  (heir  elements. 
Predictions  made  on  this  basis  have  usually  been  close  to  the  obtained 
validities  of  the  job  analogy  tests  but  have  consistently  been  under¬ 
estimates.  Our  inadequate  knowledge  of  the  factorial  content  of  such 
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tests  limits  the  conclusivencss  of  tins  evaluation.  As  mentioned  be¬ 
fore,  the  number  of  factorially  pure  tests  is  inadequate  and  there  is 
no  doubt  that  many  important  factors  have  not  been  identified.  Con¬ 
sequently,  the  factorial  content  of  a  relatively  simple  job  analog- 
test,  not  to  mention  a  criterion  job,  cannot  be  thoroughly  defined. 

Basis  of  Present  Definition  of  the  Radar  Observer’s  Job 

The  delineation  of  the  radar  observer’s  job  which  follows  consists 
of  n  job  description  paralleled  by  a  job  analysis.  Both  the.  description 
and  the  analysis  were  prepared  to  serve  as  a  basis  for  selecting  and 
constructing  tests  t  >  measure  aptitude  for  the  job  of  radar  observer.  It 
wus  recognized  that  the  selection  research  program  could  begin  with  a 
job  description  and  apply  job  analogy  tests,  or  it  could  begin  with  a 
job  analysis  and  apply  factor  tests.  Because  of  the  theoretical  interest 
in  the  relative  merits  of  the  two  approaches,  it  was  decided  to  follow 
both.  The  selection  test  program  which  resulted  is  described  in  chap¬ 
ter  11.  The  present  chapter  is  concerned  only  with  the  application 
of  the  two  methods  of  describing  a  job.-— 

In  writing  the  job  analysis,  the  problem  arose  of  choosing  a  psycho¬ 
logical  terminology.  Where  practicable,  it  was  decided  to  make  use 
of  factor  analysis  studies  and  findings  accomplished  in  the  Aviation 
Psychology  Program.  Those  analyses,  which  were  based  on  both  air¬ 
crew  classification  tests  and  experimental  tests,  had  produced  unam¬ 
biguous  empirical  evidence  of  the  existence  of  a  number  of  factors 
and  yielded  tentative  evidence  for  others.  The  test  l>attcrL„  analyzed 
have  not  included  all  types  of  measures  and  have  been  deficient  espec¬ 
ially  in  personality  and  motivational  tests.  Consequently,  it  was  be¬ 
lieved  unwise  to  depend  wholly  upon  factors  isolated  from  such 
batteries  in  attempting  to  explain  the  radar  observer’s  job.  When  these 
factors  appeared  to  be  inadequate  to  explain  a  certain  ability,  the 
analysis  was  made  in  terms  of  hypothetical  variables.  In  these  in¬ 
stances,  the  attempt  was  made  to  describe  hypothetical  variables  which 
were  “testable”  and  independent. 

In  order  to  provide  an  understanding  of  the  functions  and  abilities 
represented  by  the  factor  names  used  in  the  job  analysis  sections  to 
follow,  some  of  the  factors  that  have  been  definitely  or  tentatively 
isolated  in  the.  Aviation  Psychology  Program  are  listed  and  defined 
below.  Hypothetical  factors  will  bo  defined  in  the  text  of  the  analysis 
when  they  arc  first  mentioned.  The  tests  referred  to  in  the  following 
factor  definitions  are  described  in  appendix  A. 

1.  Length  estimation,  a  poorly  defined  factor,  refers  to  the  ability 
to  estimate  lengths  without  the  aid  of  measuring  devices.  This  factor 
is  thought  to  he  measured  by  Estimation  of  Length,  CP631A. 

2.  Mechanical  experience  represents  practical  knowledge  of  mechan¬ 
ical  devices  such  as  might  have  been  gained  through  experience  with 
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their.  This  well-defined  factor  is  best  measured  by  Mechanical  In¬ 
formation,  ClOOoB,  which  Inis  a  loading  of  about  0.75  and  Mechanical 
Principles,  CIOOGB,  with  a  loading  of  approximately  O.GOd 
3.  Memory  I  (paired  associates  memory  or  rote  memory),  a  poorly 
defined  factor,  is  thought  to  be  measured  by  Memory  for  Landmarks, 
CI510AX2.  An  almost  identical  form,  CI510AX1,  has  a  loading  of 
0.00  in  Memory  I. 

■I.  Memory  //,  or  visual  memory,  describes  the  ability  to  recognize 
previously  seen  patterns.  Although  there  is  little  evidence  for  this 
factor,  Visual  Memory,  CI5HA,  is  thought  to  measure  it  adequately. 
A  roughly  similar  test,  Map  Memory,  CI505AX2,  has  a  loading  of  0.G0 
in  Memory  II. 

5.  Numerical  facility  describes  the  ability  involved  in  carrying  out 
simple  arithmetic  computations.  It  is  also  involved  to  a  considerable 
extent  in  simply  locating  and  observing  numbers.  The  purest  measure 
of  this  well-defined  factor  is  believed  to  be  Numerical  Operations, 
CI702BX1.  An  older  form,  Numerical  Operations,  CI702A  and  B. 
has  a  loading  of  about  0.80.  Mathematics  A,  CI702F,  and  Mathe¬ 
matics  B,  CI20GC,  have  loadings  of  approximately  0.50. 

G.  The  perceptual  speed, or  identification  factor,  is  the  ability  to  note 
quickly  and  discriminate  details  in  visual  patterns.  Speed  of  Identi¬ 
fication,  CPG10C,  is  probably  the  purest  test  of  this  factor  with  an 
estimated  loading  of  0.65.  An  older  form,  CPG10B,  using  aircraft 
silhouettes  as  subject  matter,  has  a  loading  of  approximately  0.G5. 
Spatial  Orientation  I,  CP5G1B,  has  a  loading  of  approximately  0.60. 

7.  Pilot  interest  refers  to  interest  in  aviation  such  as  might  have 
been  gained  through  contact  with  it  as  a  hobby,  through  model  con¬ 
struction,  or  reading.  General  Information,  CE505E,  is  tho  best 
measure,  having  a  loading  of  approximately  0.40. 

8.  Psychomotor  coordination ,  tho  only  factor  found  to  be  charac¬ 
teristic  of  psychomotor  tests  alone,  deals  with  gross  motor  coordina¬ 
tion.  It  is  best  defined  by  Complex  Coordination,  CM701A,  with  a 
loading  of  about  0.40  and  by  Rotary  Pursuit,  CP410B.  E orm  CM703A 
of  Rotary  Pursuit,  which  did  not  require  divided  attention,  has  a 
loading  of  0.55  in  this  factor. 

0.  Psychomotor  precision  is  best  defined  by  Discrimination  Reac¬ 
tion  Time,  CP011D,  and  Finger  Dc.Verity,  CM116A. 

10.  Reasoning  /,  or  general  reasoning,  is  one  of  tho  three  reasoning 
factors,  the  others  being  too  inadequately  defined  for  use.  in  the  present 
analysis.  Spatial  Reasoning,  CI211BX2,  is  thought  to  be  an  adequate 
measure  of  this  factor  with  a  loading  of  approximately  0.55.  Mathe¬ 
matics  B,  CI20GC,  has  a  loading  of  about  0.50  but,  in  addition,  has  a 
similar  loading  in  numerical  facility  factor. 


’The  factor  loadings  presented  tn  tbls  section  nro  tnVon  primarily  from  analyses  of  the 
July  1013  and  November  UM3  nlr-crcw  elasslQcatlon  batteries. 

I 
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11.  Spare  /,  or  spatinl  rotations,  refers  to  the  ability  to  move  one’s 
self  mentally  in  si)aec  and  predict  the  result  of  such  movement  in 
ten  ns  of  position,  view  of  terrain,  etc.  It  is  best  defined  by  Instru¬ 
ment  Comprehension  II,  CIG1GB,  with  a  loading  of  approximately 
0.50.  Aerial  Orientation,  CP520A,  and  Flight  Orientation,  CP528A, 
are  thought  to  have  high  loadings  on  this  factor  although  this  has  not 
been  tested  in  analyses.  Discrimination  Reaction  Time,  CPG11D, 
and  Complex  Coordination,  CM701A,  have  loadings  near  0.40  and 
0.50,  respectively. 

12.  Spoce  //,  or  rotational  space,  identified  only  tentatively,  is 
thought  to  be  measured  by  Position  Orientation,  CP526A.  This  test 
is  a  revision  of  Thurstonc’s  Hands  Test,  CP512,  which  has  a  loading 
of  0.45  on  this  factor. 

13.  Verbal  comprehension ,  the  ability  to  understand  printed  verbal 
material,  is  best  measured  by  Air  Corps  Vocabulary  (1942)  which  has 
a  loading  of  approximately  0.70  and  Reading  Comprehension,  CI614H, 
which  has  a  loading  of  about  0.60. 

14.  Visualization  is  the  ability  to  predict  the  result  of  manipulating 
or  moving  objects  in  space  by  visualizing  the  manipulation  or  move¬ 
ment.  Pattern  Comprehension,  CP803A,  is  thought  to  have  a  loading 
of  approximately  0.50  on  this  factor,  and  Area  Visualization,  CP715A, 
is  expected  to  have  a  loading  of  about  0.50.  Mechanical  Principles, 
CI903B,  has  a  loading  of  about  0.50,  but  is  highly  loaded,  also,  on  the 
mechanical  experience  factor. 


JOB  DESCRIPTION  AND  ANALYSIS  OF  RADAR  OBSERVER 

STUDENTS  IN  TRAINING 

Extent  of  Job  Description  and  Analysis 

This  7‘ob  description  and  analysis  is  based  upon  observation  of  radar 
obscrvei  students  in  training  at  Langley  Field,  Va.  The  radar  sets 
used  in  this  training  wore  the  AN/APS-15  and  AN/APS-15A. 
Observers  watched  numerous  training  sessions,  talked  with  both  in¬ 
structors  and  students,  and  personally  completed  the  entire  training 
course. 

The  application  of  this  analysis  to  the  tasks  required  of  radar 
observers  in  combat  or  using  different  equipment  has  limited  validity. 
This  is  not  a  serious  limitation  of  the  analysis  since  its  primary  pur¬ 
pose  was  to  servo  as  a  basis  for  selecting  and  constructing  tests  to 
predict  success  in  training.  A  brief  description  of  the  job  of  the 
radar  observer  in  combat  follows  later  in  the  chapter.  The  rapidity 
of  equipment  changes  and  improvements  indicates  that  both  descrip¬ 
tions  will  be  obsolete  within  a  year.  For  this  reason  the  last  section  of 
this  chapter  presents  a  discussion  of  changes  which  may  be  expected 
in  the  radar  observer’s  job  in  the  near  future. 
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'Liu.'  radar  observer’s  job  will  be  discussed  under  four  bendings: 
Set.  operation,  scope  interpretation,  bombing,  and  navigation.  Each 
discussion  will  consist  of  a  job  description  and  a  job  analysis.  These 
will  be  presented  following  a  brief  introduction  to  the  equipment. 

Introduction  to  Air  Borne  Radar  Equipment 
Training  missions  were  carried  out  in  B-17  and  B-21  typo  aircraft : 
in  both,  the  quarters  were  cramped  and  the  table  working  surface 
provided  the  operator  is  very  limited.  Figure  4.1  shows  a  typical  in¬ 


stallation  of  the  AN/APS-15  in  the  B-17.  The  AN/APS-15  and 
AN/APS-15A  are  alike  except  for  relatively  minor  details.  The 
“eyo”  of  a  radar  set  of  this  typo  is  a  directional  antenna  mounted 
in  a  protecting  radomo  beneath  the  aircraft.*  The  antenna 
rotates  at  2G  revolutions  per  minute.  As  it  rotates,  a  transmitting- 
receiving  system  alternates,  at  the  rate  of  several  hundred  times  per 
second,  between  transmitting  short  bursts  of  high  frequency  radio 
energy  and  receiving  the  reflected  energy  or  echoes.  Each  pulse  is 
beamed  toward  the  earth  in  a  narrow  pattern  so  that  as  the  antenna 
rotates,  as  shown  in  part  II  of  figure  4.2,  successive  pulses  cover  ad¬ 
jacent  narrow  strips  of  the  terrain  radiating  from  the  point  beneath 
the  aircraft.  Thus,  a  huge  circular  area  of  the  terrain  is  scanned  ns 
the  antenna  makes  each  rotation. 

The  energy  reflected  from  the  terrain  is  translated  by  the  receiving 
and  presentation  circuits  of  the  set  into  a  picture  of  the  terrain  below 

5  The  nntciinn  and  rndomc  are  shown  In  flguro  4.3. 
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tlic  aircraft.  The  picture  is  presented  on  an  oscilloscope  known  as  the 
pi  m  petition  indicator  or  PP1  scope.  The  echoes  of  reflected  energy 
are  received  by  the  antenna  and  presented  on  the  scope  as  an  illumi¬ 
nated  rotating  radius,  called  the  sweep.  Actually  the  swing  is  a  stream 
of  electrons  which  rapidly  plays  across  the  surface  of  the  scope  begin¬ 
ning  at  the  center  and  moving  outward  radially  as  shown  in  part  III 
of  figure  1.2.  It  appears  as  a  constantly  illuminated  rotating  radius 
because  the  movement  of  the  stream  is  much  too  rapid  to  he  seen.  The 
position  of  the  sweep  is  rotated  in  phase  with  the  rotation  of  the 
antenna  as  shown  in  part  IV  of  figure  4.2.  As  the  electrons  strike  the 
surface  of  the  scope,  which  is  coated  with  a  fluorescent  substance,  they 
create  spots  or  returns  which  persist  for  a  short  time  and  vary  in 
brightness  depending  upon  the  number  of  electrons  striking  tho  coat¬ 
ing.  In  n  complete  rotation,  the  entire  circular  surface  of  tho  scope 
brightens  as  it  is  sprayed  in  rapid,  successive,  radiating  movements, 
illustrated  in  part  V  of  figure  4.2. 

Tho  various  terrain  features  such  as  mountains,  towns,  and  lakes, 
reflect  characteristically  different  amounts  of  energy.  Also,  energy 
reflected  from  distant  objects  is  received  later  and  less  strongly  than 
returns  from  near  objects.  As  the  stream  of  electrons  begins  each 
outward  radial  movement,  tho  number  of  electrons  projected  on  the 
scope  surface  is  governed  by  tho  strength  of  tho  echoes  received  from 
near  objects.  Reflected  energy  from  more  distant  objects  illuminates 
tho  sweep  farther  toward  the  edge  of  tho  scope.  As  tho  sweep  ro¬ 
tates,  always  controlled  by  the  amount  of  reflected  energy  and  the 
relative  timo  it  is  received,  a  circular  picture  of  tho  terrain  below 
tho  aircraft  is  produced  with  some  terrain  features  appearing  as 
bright  areas  and  others  appearing  as  dark  areas.  The  appearance  of 
the  scope  picture  is  shown  in  part  V  of  figure  4.2  and  figure  4.3.  Tho 
center  of  tho  picture  represents  tho  point  directly  below  tho  aircraft, 
and  distant  objects  arc  presented  toward  the  edge  of  tho  scopo. 

An  auxiliary  unit  of  tho  set  can  be  made  to  project  bright  concen¬ 
tric  circles  on  tho  scopo  marking  off  uniform  and  known  distances 
from  tho  center.  These  range  marks,  together  with  an  azimuth  scale, 
graduated  in  degrees  around  tho  edge  of  the  scope,  convert  tho  pic¬ 
ture  into  a  polar  grid  map  on  which  it  is  possiblo  to  express  tho  posi¬ 
tion  of  any  point  in  terms  of  distance  from  tho  center  and  bearing 
in  degrees  in  azimuth.4  An  additional  and  more  precise  range  meas¬ 
uring  device,  tho  range  unit,  enables  the  radar  observer  to  set  a 
bright  circle,  the  bomb  release  circle,  on  tho  scopo  accurately  at  any 
given  distance  from  tho  aircraft’s  ground  position. 

A  second  scope,  tho  A  scopo,  presents  targets  ns  vertical  pips  and 
is  used  for  tuning  and  calibrating  tho  set.  Tho  controls  which  the 

4  The  appearance  of  tho  scopo  with  r.ingo  marks  and  azimuth  calibration  Is  shown  In 
flgur*  4.4. 
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radar  observer  must  manipul:  to  avo  distributed  over  tho  several  units 
of  tho  set. 

Set  Operations  Jol>  Description 

Preope rational  check. — Prior  to  turning  on  tho  set,  the  controls 
must  bo  positioned  correctly.  This  prevents  damaging  parts  of  tho 
f.et  by  the  initial  surge  of  electricity  from  tho  aircraft’s  power  supply, 
prevents  overloading  tho  power  supply,  and  places  tho  controls  in 
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known  positions  that  will  facilitate  1 1  if*  subsequent  starling  and  tun 
ir.g  of  the  set.  The  prooperational  check  consists  of  checking  lor 
loose  connections,  checking  for  incorrect  and  defective  fuses,  and 
checking  the  positions  of  about  20  switches,  knobs,  and  screw-driver 
adjustments.  Tho  check  is  ordinarily  carried  out  before  the,  aii  craft 
takes  off. 

fiturllnij  (i/nf  hin/nr/. — To  st art  the  set  properly  and  without  damage, 
certain  controls  must  be  turned  and  adjusted  in  a  prescribed  manner. 
Starting  includes  applying  power  to  the  set ,  and  properly  brightening, 
focusing,  and  centering  the  sweeps  on  both  the  PPI  scope  and  the  A 
scope.  '1'h is  consists  of  about  20  steps,  most  of  which  have  to  be 
performed  in  a  given  sequence.  The  radar  observer  checks  his  adjust¬ 
ments  by  reading  a  current  meter  and  by  watching  the  bright  lines 
presented  on  the  PPI  scope  and  the  A  scope. 

Tuning  consists  >f  turning  on  the  transmitter,  picking  up  ground 
returns  and  adjusting  the  transmission,  receiving,  and  presentation 
systems  for  maximum  definition  on  the  PPI  and  A  scopes.  About 
20  steps  must  be  performed  in  sequence  to  tune  the  set  according  to 
standard  operating  procedure.  During  the  tuning,  the  radar  observer 
adjusts  for  specific  or  maximum  current  meter  readings,  and  for 
maximum  contrast  between  target  returns  and  other  ground  returns. 
On  the  A  scope,  this  contrast  is  in  terms  of  height  of  bright  returns, 
or  “pips'5  above  a  base  line.  On  the  PPI  scope,  target  returns  are 
brighter  than  other  ground  returns.  Starting  and  tuning  are  carried 
out  nfter  tho  aircraft  has  taken  off  and  reached  an  altitude  of  about 
1,000  feet. 

Calibration. — The  range  unit  is  a  device  for  precise  measurement 
of  distance.  Its  accurate  functioning  is  essential  to  bombing  and  pre¬ 
cision  navigation.  Calibration  consists  of  checking  the  range  unit 
against  a  standard  measuring  scale  built  into  the  set  and  presented  on 
the  A  scope  ns  a  line  divided  into  20  units.  The  radar  observer  manip¬ 
ulates  knobs  and  screw-driver  adjustments,  making  a  very  fine  ad¬ 
justment  until  nil  points  on  the  two  scales  are  lined  up.  The  process 
involves  lo  to  00  steps,  depending  on  the  set  and  the  operator’s  skill, 
Mo.st  of  the  steps  must  be  performed  in  prescribed  sequence.  Since 
the  calibration  of  the  set  is  affected  by  altitude  and  temperature,  it  is 
usually  done  after  the  plane  has  reached  the  altitude  at  which  the 
mission  is  to  be  flown. 

I'Vujht  maintenance. — The  success  of  a  radar  mission  may  depend 
upon  the  radar  observer’s  ability  to  correct  certain  set  malfunctions 
during  flight.  The  radar  observer  must  be  able  to  diagnose  what  is 
wrong  with  the  set  from  the  particular  circuits  that  arc  inoperative, 
from  the  various  meter  readings,  or  from  the  picture  presented  on  the 
scopes.  Common  troubles  are  blown-out  fuses,  looso  cables,  and  varia- 


30 


lions  in  ( In*  aircraft’s  power  supply.  Tlu*  radar  observer  must,  know 
w  hat  tiling  to  check  when  specific  symptoms  are  noticed. 

Maintai niiit/  art  c/ffeintey. — Continual  adjustment  is  necessary  to 
insure  that  the  set  remains  properly  calibrated  and  tuned.  Changes 
in  temperature,  altitude,  power  supply,  weather  conditions,  and  ter¬ 
rain  all  have  their  cll'eet.  Usually  such  variations  will  not.  mako 
navigation  and  bombing  impossible,  hut  will  elTret  accuracy  consider¬ 
ably.  If  he  is  to  obtain  maximum  operating  efficiency  the  radar  ob¬ 
server  should  be  aware  of  possible  changes  and  make  periodic  checks 
on  the  power  supply,  tuning,  and  calibration. 

Turning  off  set. — The  process  of  turning  the  set  olf  is  simple,  con¬ 
sisting  of  only  seven  steps.  The  only  restriction  is  that  four  brilliance 
controls  must  be  turned  down  before  the  power  switch  is  turned  oil 
to  prevent  burning  the  scopes. 


Set  Operation:  Job  Analysis 

In  learning  to  operate  the  set,  the  radar  observer  student  must  study 
technical  material,  memorize  the  steps  in  the  operating  procedures, 
and  know  the  location  and  function  of  the  various  set  controls.  This 
learning  involves  memory  1 B  and  verbal  comprehension. 

Tie  student  who  develops  a  rationale  for  the  operating  procedures, 
t ho  functions  of  the  controls,  and  the  relationship  between  tho  parts 
of  the  set  will  probably  learn  set  operation  more  easily  and  retain 
the  material  longer.  Even  though  his  rationale  is  not  technically 
correct,  it  will  aid  him  in  remembering  which  step  is  next  in  tuning, 
which  fuse  to  check  if  a  certain  circuit  goes  out,  and  which  controls  to 
manipulate  to  improve  the  scope  picture.  This  ability  can  probably 
ho  measured  by  tests  of  “scientific  information”  and  tests  of  experi¬ 
ence  with  electrical  and  meclmical  contrivances.  Tho  aircrew  classi¬ 
fication  tests  loaded  with  the  mechanical  experience  factor  should  also 
constitute  fairly  adequate  measures  of  this  aspect  of  the  job.  Some 
reasoning  /  (general  reasoning)  may  be  involved  hero  also. 

The  radar  observer  who  is  thorough  and  systematic  will  operate  tho 
set  more  carefully  and  check  the  tuning  and  calibration  more  fre¬ 
quently.  'Ibis  quality  of  work  may  be  related  to  habits  of  organiza¬ 
tion  and  thoroughness  as  shown  in  the  individual’s  work  ami  hobby 
history.  The  same  quality  has  been  designated  “systematic  diligence” 
elsewhere. 


Some  radar  observers  exhibit  “finger  trouble”  in  timing  the  set. 
They  const  ant  1\  manipulate  the  controls  trying  to  improve  the  pictnro 
and  neglect,  other  tasks.  Such  men  are  never  sure  of  having  obtained 
the  best,  presentat  ion  possible  under  prevailing  conditions.  This  may 


1  In  tlila  and  lntor  Job  analysis  sections,  nil  factor  names  liavo  been  underlined  or  placed 
■nltldn  i)imlntlori  marks.  Underlining  lndlcateii  tlint.  the  factor  lias  been  Isolated  In  factor 
analyse*  in  Hie  Avlnllon  PsyclioloKy  Program.  All  others,  hypothetical  or  Isolated  else- 
"here,  linv«  been  placed  within  <]uolntIon  marks. 
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indicate  poor  visual  memory,  /armory  II ,  since  the  radar  observer 
must  remember  the  be-t  picture  obtained  (lining  the  control  adjust¬ 
ments  and  finally  reproduce  it.  lit*  must  also  remember  the  appear¬ 
ance  of  scope*  pictures  produced  by  his  instructors  under  various  con¬ 
ditions.  “Finger  trouble1’  may  also  indicate  a  lack  of  confidence  in 
one’s  operating  ability.  ' 

Psychomotor  precision  is  involved  in  manipulating  the  switches, 
knobs,  and  screw-driver  adjustments.  Fairly  line  adjustments  of  the 
rotatable  controls  are  required  but  little  speed  is  necessary. 

“Visual  acuity”  and  “brightness  discrimination”  are  required  in 
observing  the  scopes,  dial*-,  and  neon  light,  and  particularly  in  detect¬ 
ing  the  near-threshold  length  and  brightness  changes. 

Scope  Interpretation:  Job  Description 

Interpreting  returns. — Tbo  features  of  the  terrain  are  represented 
ns  characteristic  returns  which  const  itute  the  picture  on  the  PPI  scope. 
Tho  radar  observer  selects  specific  returns  to  determine  bis  ground 
position,  to  find  the  target  area,  and  to  detect  within  the  target  area 
tho  aiming  point  to  be  bombed.  Unless  the  returns  are  correctly  inter¬ 
preted  and  identified  on  the  map,  all  the  radar  observer’s  skills  in  set 
operation,  navigation,  and  bombing  are  of  no  valuo. 

As  shown  in  fignro4.3,  returns  on  the  scope  differ  in  brightness,  size, 
and  shape.  Water  gives  practically  no  return.  Islands  on  water  ap¬ 
pear  as  bright  areas  against  a  dark  background.  Built-up  areas  on 
land,  such  as  towns,  appear  ns  bright  returns  against  a  less  bright  back¬ 
ground.  Rivers,  lakes,  and  inlets  appear  as  dark  areas  with  bright 
lines  representing  tho  far  shores.  Mountain  ranges  appear  as  long 
bright  areas  with  shadows  behind  them.  Towns  behind  mountain 
ranges  or  in  valleys  are  hidden  except  when  approached  from  certain 
angles.  Dense  clouds  appear  as  bright  areas  and  usually  have  com¬ 
plete  shadows  behind  them.  Bridges  appear  ns  sharp  bright  lines 
against  the  dark  water.  Towns,  islands,  rivers,  lakes,  etc.  retain  some¬ 
thing  of  their  shape,  but  details  are  usually  lost  or  distorted. 

The  appearance  of  returns  from  the  various  terrain  features  must 
he  learned  and  remembered  by  the  radar  observer.  Because  these  differ 
considerably,  the  task  of  deciding  whether  a  return  represents  a  river, 
lake,  town  or  mountain  is  usually  not  difficult.  Only  occasionally  is 
tliero  some  problem,  as  for  example,  deciding  whether  a  return  is  a 
dense  cloud  or  a  town.  However,  specifically  identifying  a  return  once 
it  has  been  recognized  as  a  town  is  more  difficult.  By  noting  the  size, 
shape,  and  brightness  of  the  return,  the  radar  observer  is  sometimes 
ublo  to  identify  or  name  it  without  reference  to  other  returns.  Fre¬ 
quently,  however,  none  of  the  returns  from  towns  have  characteristic 
shapes  or  sizes  ami  no  other  characteristic  terrain  returns  are  available. 
In  such  instances  where  the  specific  returns  arc  homogeneous  and  with- 
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out  indi viduality,  identification  is  po.Wible  only  by  t lie  pattern  which 
(In1  returns  form  in  terms  of  distances  uml  ln*a rinj^s  from  each  other. 

Identification  of  aiming  points  within  towns  or  target  areas,  which 
is  crucial  to  accurate  bombing,  is  a  very  difficult  task.  The  various 
areas  within  large  cities  can  usually  be  distinguished  on  the  scope  if 
the  set  is  operated  correctly.  For  example,  factory  and  business  areas 
ordinarily  give  brighter  returns  than  residential  areas.  Railroad 
yards  are  brighter  than  surrounding  areas.  The  radar  observer  must 
remember,  from  the  scope  photographs  used  in  briefing,  the  shape  of 
the  aiming  point,  how  it  contrasts  with  the  rest  of  the  target  area,  and 
its  position  relative  to  other  elements  in  the  target  area.  lie  must 
operate  the  set  skill  fully  enough  to  define  the  aiming  point  on  the  scope 
and  he  must  recognize  it  from  the  size,  shape,  brightness,  and  pattern 
cues.  This  identification  must  be  done  rapidly  since  the  target  area  on 
the  scope  breaks  up  into  its  elements  only  during  the  last  5  or  10  miles 
of  the  bomb  run  which  is  covered  in  from  two  to  three  minutes. 

Difficulties  in  interpreting  and  identifying  returns  depend  in  part 
upon  the  radar  observer’s  other  skills.  Poor  set  operation  will  cause 
returns  to  be  poorly  defined  or  lost  altogether.  If  the  radar  observer 
does  bis  navigation  calculations  rapidly,  he  will  have  more  timo  to 
watch  the  scope.  As  new  returns  appear  on  the  scope  he  will  be  able 
to  identify  them  by  referring  to  returns  he  already  has  identified.  Con¬ 
versely,  the  radar  observer  who  is  slow  in  computational  work  will  fre¬ 
quently  be  faced  with  the  task  of  interpreting  and  identifying  a  com¬ 
pletely  unfamiliar  pattern  of  returns. 

Scope  interpretation  is  facilitated  not  only  by  rapid  but  by  accurate 
navigation.  Radar  observers  who  do  nceuratc  navigation  know  their 
approximate  ground  position  and  course  and  can  predict  which  towns 
and  terrain  features  are  about  to  enter  the  range  of  the  set  and  where 
the  returns  will  appear  on  the  scope.  F rom  this  information,  they  aro 
able  to  identify  individual  returns  when  they  are  distorted  and  when 
parts  of  patterns  of  returns  are  not  visible. 

When  the  azimuth  stabilization  unit  of  the  set  is  “on,”  the  top  of 
tho  scope  picture  is  true  north.  The  returns  are  in  the  same  relative 
positions  in  which  they  appear  on  the  map.®  When  azimuth  stabiliza¬ 
tion  is  “off,”  the  top  of  the  scope  represents  the  direction  in  which  the 
aircraft  is  heading.  If  the  aircraft  is  heading  in  any  direction  other 
than  true  north  the  pattern  of  returns  is  rotated  from  the  po  it  ion  of 
the  corresponding  towns  on  the  map.  This  rotation  increases  the 
difficulty  of  identifying  patterns  and  targets. 

Interpreting  motion  of  returns .  The  radar  observer  can  set  an 
illuminated  radius,  the  lubber-line,  on  the  PPI  scope  to  indicate  the 


•Tho  relation  between  the  (scope  picture  and  n  map  wi  n  azimuth  ntublllrutlon  is  "on" 
I'hown  In  figure  -l.-l.  This  relationship  would  be  the  name  re  Kurd  I  css  of  the  heading 
of  the  aircraft. 
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direction  (lie  :i i re i  ;i  ft  is  heading.  By  ob. erring  (Id1  movement  of  re- 
linns  in  volution  to  the  lubber  line,  lie  can  determine  the  direction  in 
which  the  [thine  is  drifting  or  being  deflected  from  its  bending  by  the 
wind.  If  the  aircraft  is  not  drifting  at  nil,  the  returns  will  move 
para  11  (d  to  the  heading  of  the  nicer  a  ft.  If  the  airern  ft  is  drifting  to 
the  right ,  ret  urns  will  move  from  the  right  to  the  left  of  the  lnl>bor-Iine. 

Tn  navigation  and  bombing,  the  radar  observer  interprets  the  move¬ 
ment  of  ret  m  ns  on  the  scope  in  terms  of  direct  ion  in  which  the  aircraft 
is  drifting.  In  navigation,  the  direction  of  drift  gives  him  a  check 
upon  his  wind  computation  and,  if  measured  accurately,  can  be  used  to 
compute  a  wind.  In  bombing,  the.  radar  observer  “kills  drift”  on  the 
target  by  giving  heading  corrections  until  the  target  is  moving  neither 
to  the  left  nor  to  the  right  of  the  heading  line.  This  inlerpretational 
problem  is  most  dillicnlt  when  the  aircraft  is  heading  toward  the  bot¬ 
tom  of  the  scope.  Then  the  radar  observer’s  right  is  the  aircraft’s  left 
and  vice  versa.  Here  it  is  important  that  he  interpret  the  motion  of  the 
returns  relative  to  the  aircraft’s  heading  and  not  relative  to  his  view 
of  the  scope. 

Scope  Interpretation:  Job  Analysis 

Identification  of  returns  from  shape,  size,  brightness,  and  pattern 
seems  to  be  related  most  closely  to  the  perceptual  speed  and  memory  II 
(visual  memory)  factors.  Perceptual  speed  is  important  in  the  task  of 
locating  map  features  which  arc  similar  to  the  scope  returns  and/or 
in  locating  scope  returns  which  are  similar  to  points  on  the  map. 
Memory  II  seems  important  in  recognizing,  on  the  scope,  patterns  of  re¬ 
turns  previously  seen  on  the  map  or  during  the  briefing  session  and  in 
remembering  the  characteristic  appearance  of  the  various  types  of 
terrain.  In  remembering  and  matching  patterns  of  returns,  length 
estimation  as  well  as  the  abilities  to  estimate  sizes  and  angles  are  prob¬ 
ably  important.  All  patterns  are  composed  of  various  sized  returns  at 
different,  distances  aiul  directions  from  each  other. 

It  is  likely  that  the  student  who  has  had  scientific  training  and  ex¬ 
perience  can  understand  more  easily  why  the  various  terrain  features 
yield  charaeterbtic  returns.  Therefore,  “scientific  background”  would 
probably  be  an  aid  to  remembering  and  identifying  returns  from  in¬ 
dividual  targets. 

Perceptual  speed  and  memory  II  seem  particularly  important  in 
identifying  the  aiming  point.  Since  the  identification  must  bo  done 
rapidly,  the  radar  observer  must  rely  on  his  memory  of  the  target 
area;  he  has  little  time  to  consult  scope  photographs.  The  pattern  of 
returns  is  often  only  dimly  visible  or  is  partial!  obernred.  Fine 
bright  mss  discriminations  are  often  required  even  t^  ^eix^vo  the  pat¬ 
tern. 

Perceiving  and  interpreting  the  motion  of  returns  across  the  scope 
seem  to  depend  upon  ability  in  memory  II  and  space  I  (spatial  rola- 
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(ions).  The  returns  usually  move  across  the  scope  at  an  almost  im¬ 
perceptible  rate.  The  radar  observer  cannot  take  time  to  watch  the 
scope  for  any  lengl  h  of  time  to  determine  the  direct  ion  of  mot  ion  of  the 
returns.  Instead  he  must  look  at  the  scope  at  intervals.  Memory  II 
is  necessary,  since  each  time  he  looks  at  the  scope,  he  must  remember 
where  the  returns  previously  were  in  order  to  determine  the  direc¬ 
tion  they  moved. 

When  the  aircraft  is  heading  in  a  southerly  direction,  with  azimuth 
stabilization  “on,’’  the  radar  observer  must  imagine  himself  rotated  in 
space  to  make  his  light  and  left  correspond  to  the  aircraft’s  right  and 
left.  This  will  enable  him  to  interpret  correctly  whether  the  aircraft 
is  dri  fting  to  the  right  or  left.  lion  azimuth  stabilization  is  “off,”  this 
same  ability  to  imagine  one’s  self  rotated  in  space  is  necessary  in  all 
headings  except  true  north.  This  operation  is  similar  to  that  required 
in  tests  which  have  loadings  in  space  I. 

Insofar  as  scope  interpretation  is  dependent  upbn  speed  of  compu¬ 
tational  work  and  quality  of  set  operation,  ability  here  will  be  related 
to  the  abilities  described  in  the  job  analyses  of  set  operation,  above, 
and  navigation,  below. 

Navigation:  Job  Description 

Flight  planning. — Before  a  mission,  the  radar  observer  obtains  the 
■following  information  :  route,  forecast  or  “metro”  wind,  altitude,  tem¬ 
perature  forecast  for  the  altitude,  and  indicated  airspeed.  On  a.  map, 
lie  draws  in  the  courses  for  the  navigation  legs  and  bomb  runs.  lie 
measures  the  direction  of  each  leg  (true  course  or  track  to  make  good) 
using  the  dividers  and  the  protractor  scale  of  live  Weems  plotter.  lie 
measures  the  length  of  each  leg  by  first,  stretching  the  dividers  over  the 
leg  and  then  laying  it  out  along  the  degrees  and  minutes  scale  on  the 
edge  of  the  map,  one  minute  of  latitude  being  equal  to  one  nautical 
mile. 

The  radar  observer  converts  indicated  airspeed  from  statute  miles 
per  hour  to  nautical  miles  per  hour  using  tho  slide  rule  scale  of  the 
E-GB  computer.  Then  setting  pressure  altitude  opposite  forecast 
temperature  for  that  altitude  on  the  E-GB  computer,  he  reads  true  air 
speed.  lie  draws  the  wind  force  and  wind  velocity  on  the  vector  face 
of  the  E-(iB,  centers  the  vector  face  on  the  true  air  speed,  and  deter¬ 
mines  what  direction  the  aircraft  would  drift  if  headed  on  the  true 
course.  lie  then  “juggles”  the  computer  until  the  drift  would  cause 
the  aircraft  to  travel  along  the  true  course.  From  the  computer  lie 
reads  the  true  heading  which  the  aircraft  should  take  in  order  to  make 
good  the  true  course  and  the  ground  speed  that  the  aircraft  will  mako 
on  that  heading.  On  the  slide  rule  scale  of  the  E-GB  computer  he  de¬ 
termines  how  long  it  will  take  to  fly  each  leg,  Inn  ing  all  uuly  found  the 
distance  and  ground  speed. 
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All  of  tin;  information  <h\>-cribod  above,  both  given  ami  computed,  is 
entered  in  the  (light  plan  section  of  the  log.  From  this  (light  plan,  the 
radar  observer  can  tell  the  pilot ,  on  the  basis  of  the  met  ro  wind  and  the 
other  predicted  data,  what  heading  to  lake  for  each  leg  and  how  long 
it  will  take  to  fly  the  leg. 

Maintaining  record  of  ground  position  and  flight  data.-- One  of  the 
most  important  functions  of  the  radar  observer  is  to  provide  ground 
position  reports.  This  function  is  invaluable  at  night  or  when  the 
aircraft  is  over  clouds.  The  area  represented  on  the  PPI  scope  is  a 
polar  coordinate  map,  the  center  being  the  ground  position  or  point 
directly  under  the  aircraft.  Ground  position  can  bo  determined  by 
identifying  one  or  more  returns  on  the  scope,  determining  the  spatial 
relationship  between  the  center  of  the  scope  and  the  returns,  and 
plotting  this  relationship  on  the  map  starting  with  the  points  cor¬ 
responding  to  the  returns  and  working  back  to  the  ground  position 
corresponding  to  the  scope  center.  Plotting  ground  position  in  this 
manner  is  called  “taking  a  fix.” 

Three  types  of  fixes  may  be  taken :  a  range  and  bearing  fix,  a  mul¬ 
tiple  bearing  fix,  and  a  multiple  range  fix.  The  first  is  more  frequently 
used.  It  consists  of  measuring  the  range  and  bearing  from  tho  centor 
of  tho  scope  to  a  single  identified  return.  A  multiplo  bearing  fix 
consists  of  measuring  only  the  bearing  from  the  center  of  tho  scope 
to  each  of  two  or  more  identified  returns.  A  multiplo  rango  fix  con¬ 
sists  of  measuring  only  the  range  from  the  center  of  the  scopo  to  each 
of  two  or  more  identified  returns. 

To  determine  tho  bearing  of  a  return,  the  radar  observer  makes 
uso  of  a  rotatable  plexiglass  face  covering  the  scope.  This  plexiglass 
is  bisected  by  an  etched  lino  which  may  bo  rotated  until  it  passes 
through  tho  return.  He  then  reads  the  bearing  at  the  point  where 
tho  lino  intersects  tho  azimuth  ring  at  the  edge  of  the  scope.  This 
value  is  the  direction  from  the  center  of  the  scopo  to  tho  return.  In 
figure  4.4,  tho  bearing  of  the  point  of  land  being  used  as  a  check  point 
is  3*28°.  Tho  range  of  a  return  may  be  measured  in  two  ways.  In 
the  most  frequently  used  method  tho  radar  observer  positions  bright 
concentric  circles  on  the  scope  at  any  one  of  the  several  intervals.  The 
usual  interval  is  5  or  10  miles.  Using  these  circles,  he  reads  the 
distance  to  a  given  return  by  estimating  to  the  nearest  mile  or  half- 
mile  between  the  marks.  In  figure  4.4,  the  check  point  is  40  miles 
out  on  the  scope.  In  the  second  method,  less  frequently  used,  he 
measures  ranges  more  precisely  with  tho  bombing  circle.  Ho  turns 
on  the  bombing  circle,  turns  a  knob  until  tho  bright  circle  expands 
and  touches  tho  return,  and  reads  the  distance  from  a  nautical  mile 
scale  or  counter  dial  calibrated  in  hundreds  of  feet.  In  tho  latter 
case,  ho  has  to  convert  to  nautical  miles  before  plotting  the  fix. 
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Fioure  4.1 — Fix-taking  from  the  FPI  scope. 

Tlie  radar  observer  plots  a  range  and  bearing  fix  by  first  plotting 
the  direction  from  the  point  corresponding  to  the  return.  Sinco  hi3 
bearing  reading  from  the  scope  was  the  direction  from  the  ground 
position  to  the  return,  be  must  plot  the  reciprocal  of  this,  the  bearing 
plus  180°,  as  shown  in  figure  4.4.  This  directional  plotting  is  done 
using  the  protractor  scale  of  the  Weems  plotter.  After  positioning 
the  plotter,  he  measures  oil  the  proper  range  on  his  dividers  from 
the  latitude  scale  at  the  edge  of  the  map  and  lays  this  distanco,  40 
miles  in  figure  4.4,  along  the  plotter.  The  resulting  point  is  the 
ground  position  of  the  aircraft  at  the  time  of  the  fix. 

To  make  possible  further  navigational  work  beyond  the  flight  plan, 
it.  is  essential  that  the  radar  observer  maintain  a  continuous  record 
of  the  aircraft’s  ground  position  and  track  mado  good.  Consequently, 
he  must  take  fixes  at  regular  intervals,  log  them,  and  plot  them.  Ho 
enters  them  in  the  log  by  recording  the  time  of  the  fix,  the  return 
used,  and  the  range  and  bearing  date.  He  also  records  flight  data 
in  the  log  at  regular  intervals,  particularly  noting  changes  in  air 
speed,  altitude,  and  true  heading.  To  obtain  this  information,  ho 
must  read  the  fluxgate  compass,  altimeter,  air  speed  indicator,  and 
free  air  temperature  gauge. 

fl  i'mi  computation. — The  winds  predicted  by  the  weather  depart¬ 
ment  have  oidy  a  limited  value  for  exact  navigation  since  they  aro 
approximations,  apply  to  limited  areas,  and  cannot  take  into  account 
sudden  wind  shifts.  One  of  the  important  tasks  of  the  radar  operator 
is  to  compute  winds  using  instrument  data  ami  data  from  his  fixes. 
Ihree  methods  of  wind  computation  will  be  described. 
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In  the  airplot  method,  from  the  fixes  entered  in  the  lug  and  plotted 
on  his  map,  the  radar  observer  can  determine  the  track  made  over 
a  given  interval  of  time  and  his  ground  position  at  the  end  of  the 
interval.  lie  plots  on  the  map  the  position  the  aircraft  would  nave 
reached  if  it  had  been  traveling  in  still  air.  This  position  is  called 
the  air  position  and  is  hown  in  relation  to  the  ground  position  in 
figure  4.5.  To  determine  the  aircraft’s  air  position  at  the  given  time, 
the  radar  observer  starts  with  a  previous  ground  position  and,  using 
the  "Weems  plotter,  plots  the  true  heading  or  course  that  would  have 
been  made  good  in  still  air.  To  determine  the  distance  that  would 
have  been  traversed  in  still  air,  he  uses  true  air  speed  on  the  K-GB 
slide-rule  scale.  lie  measures  this  distance  oil  the  side  of  the  map 


Figuiuc  ‘1.5— Determining  wind  by  airplot. 
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with  the  dividers  and  lays  it  oil  on  the  true  heading.  The  resulting 
point  is  the  air  position,  the  hypothetical  point  the  aircraft  would 
have  reached  Hying  in  still  air  on  the,  true  heading  at  the  rato  of 
the  true  air  speed  for  the  given  time  interval.  The  line  drawn,  as 
in  figure  4.5,  from  the  air  position  to  the  ground  position  at  the 
corresponding  time,  indicates  the  direction  and  distance  the  wind 
carried  the  aircraft  during  the  time  interval.  The  radar  observer 
measures  the  direction  of  this  line,  in  degrees  ir/.inmth,  with  tho  Weems 
plotter.  Tie  knows  that  the  wind  blew  the  aircraft  from  the  air  po¬ 
sition  to  the  ground  position  and  that  wind  direction  is  indicated 
in  terms  of  the  direction  from  which  it  comes.  If.:  then  measures 
the  length  of  the  line  with  the  dividers  and  divides  it  by  tho  time 
interval  on  the  slide  rule  scale  of  the  K-OB  to  get  the  wind  force. 

The  second  method  of  wind  computation  is  tho  regular  computer 
method.  As  previously  stated,  the  radar  observer  can  determine  from 
his  logged  and  plotted  fixes  the  true  course  or  track  made  good.  By 
measuring  tho  distance  between  two  fixes  and  dividing  it  by  the  time 
interval  between  the  fixes,  lie  can  obtain  the  ground  speed  for  the  true 
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(jou i  Ho  places  the  vector  face  index  of  1  lie  E-GB  computer  at  the 
(me  heading,  centers  the  face  on  the  true  air  speed,  and  plots  the. 
mound  speed  on  the  true  course.  lie  rotates  the  vector  face,  reads  the 
v.iiul  direction  from  the  azimuth  scale  and  the  wind  velocity  along  the 
mile  scale. 

The  third  method  of  wind  computation  is  the  grid  computer  method, 
•il so  called  target  timing.  The  radar  ohseivcr  takes  two  or  more,  suc¬ 
cessive  fixes  on  the  same  return  and  times  the  interval  between  the 
two  with  a  stop  watch.  lie  then  plots  the  fixes  on  the  vector  face  of 
the  E-GB  computer  over  the  square  grill.  Since  returns  travel  across 
the  scope  parallel  to  the  track  the  aircraft  is  making  good,  the  line 
connecting  these  fixes  is  parallel  to  the  aircraft's  track.  lie  rotates 
the  vector  face  until  the  connecting  line  is  aligned  with  the  vertical 
grid  lines  and  reads  the  direction  of  the  track.  At  this  point  he  must 
check  that  he  is  not.  reading  the  reciprocal  direction  by  comparing 
(lie  computer  result  with  his  general  estimate  of  the  track.  lie  divides 
the  distance  between  fixes  by  the  time  between  fixes  and  computes  an 
average  or  over-all  ground  speed.  lie  then  uses  true  course  and 
ground  speed  oil  the  vector  face  of  the  computer  to  determine  tho  wind 
by  (he  regular  computer  method  described  above. 

Planning  remainder  of  flight. — Frequent  and  aecurnto  computa¬ 
tion  of  ground  position  and  wind  enables  the  radar  observer  to  navi¬ 
gate  the  rest  of  the  mission  accurately.  Unless  the  metro  wind  is  very 
accurate,  (lie  aircraft  will  likely  fly  oil  course  and  not  make  a  track  for 
the  dost  inat  ion.  In  this  case,  the  radar  observer  uses  his  best  wind  in¬ 
formation  to  determine  what  heading  lie  should  give  the  pilot  to  cor¬ 
rect  the  track.  To  this  end  he  must  do  dead  reckoning  navigation 
along  his  track  from  the  last  ground  position  and  determine  an  ap¬ 
proximate  point  he  will  reach  several  minutes  later.  On  tho  E-GB, 
lie  computes  the  distance  he  will  travel  in  a  certain  number  of  minutes 
with  1 1 is  present  ground  speed.  lie  lays  this  distance  out  along  the 
track  and  plans  to  make  a  course  correction  at  the  resulting  point, 
lie  draws  the  true  course  from  that  point  to  the  destination  and  meas¬ 
ures  its  direction.  lie  juggle’s  the  computer,  ns  described  above,  know¬ 
ing  true  course,  true  air  speed,  wind  velocity,  and  wind  direction,  and 
deter mines  wliat  heading  will  be  necessary  to  make  good  the  true 
course.  When  the  course  correction  point  is  reached — that  is,  when  tho 
given  time  interval  lias  elapsed — he  gives  the  pilot  the  new  heading, 
lie  al  so  computes  an  estimated  time  of  arrival  for  the  destination  so 
that  he  will  know  approximately  when  lie  is  there.  This  information 
is  particularly  important  when  the  point  cannot  he  seen  or  picked  up 
on  the  scope. 

Tho  radar  observer  uses  his  latest  wind  information  in  a  similar 
manner  to  compute  headings  for  the  bomb  run  so  that  the  aircraft 
will  drift  as  little  as  possible  away  from  the  target. 
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Since  any  single  wind  coin j hi t a  1  inn  may  in*  innenirnt  e  and  (lie  wind 
may  change,  the  radar  observer  determines  new  winds  as  often  as  ho 
has  lime.  The  latent  or  ino.-t  tin  led  wind  is  n.-eil  in  computing  course 
corrections  or  headings  oil  new  legs. 

Navigation  legs  in  training  range  in  length  from  GO  lo  100  miles. 
Consequently,  the  radar  observer  has  a  limited  period  in  which  to 
compute  winds,  course  corrections,  and  new  headings.  After  allowing 
for  passage  of  snlUeiml  time  to  collect  accurate  wind  data  lie  will  have 
from  10  to  2.">  minutes  depending,  of  course,  upon  (lie  length  of  the  leg 
and  (he  wind.  During  (his  time  he  must  also  watch  the  scope,  identify 
new  returns,  and  maintain  the  ellicient  operation  of  the  set.  If  the 
navigation  leg  is  followed  by  a  bomb  mil,  lie  has  to  spend  time  near 
the  end  of  the  leg  predicting  ground  speed  and  absolute  altitude  over 
the  target  and  preparing  the  set  for  the  bombing  procedures  to  be  de¬ 
scribed  below. 

Navigation:  Job  Analysis 

One  of  the  principal  problems  confronting  the  student  on  radar 
training  missions  is  that  of  completing  the  necessary  navigational 
work  in  time,  without  having  to  do  hurried  and  inaccurate  work. 
Good  radar  observers  have  systems  for  carrying  out  their  work  by 
means  of  which  they  obtain  each  bit  of  data  as  soon  as  it  is  available 
and  spread  out  their  computational  work  as  much  as  possible.  This 
planning  ability  depends  partly  upon  already  having  thorough  under¬ 
standing  of  how  the  separate  steps  of  navigational  procedure  arc  re¬ 
lated  to  each  other.  Such  an  integrated  view  of  navigational  work  is 
probably  most  readily  obtained  by  a  student  who  has  a  background  of 
mathematics  and  science.  “Scientific  background”  and  reasoning  I 
(general  reasoning)  are  believed  to  he  the  abilities  involved  in  this  as¬ 
pect  of  navigational  work  planning. 

In  Ins  navigational  work,  the  radar  observer  reads  many  dials  and 
scales:  E-GB  computer  scales,  navigation  instruments,  and  dials  and 
scales  on  the  set.  Components  of  scale-reading  ability  seem  to  be  nu¬ 
merical  facility  and  space  I  (spatial  relations). 

In  reading  ranges  on  the  PPI  scope,  the  radar  observer  estimates  dis¬ 
tance  within  the  interval  between  the  range  marks.  Ho  makes  similar 
estimates  in  measuring  along  the  nautical  mile  scale  (minutes  of  lati¬ 
tude)  on  the  map.  A  length  estimation  ability  is  believed  necessary 
for  these  tasks. 

After  tho  radar  observer  obtains  the  navigational  data  and  plans 
how  to  use  them,  he  sots  them  into  his  computer,  plots  them  on  a 
chart,  or  sets  them  into  the  radar  equipment.  Psychomotor  precision 
is  involved  in  these  operations,  particularly  in  handling  the  dividers, 
plotter,  and  computer. 

As  the  radar  observer  carries  out  lus  plotting,  measuring,  and  com¬ 
puting,  he  should  check  the  results  of  each  step.  lie  can  do  this  by 
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m  iking  approximate  calculations  mentally.  Xiniiorical  facility  ami 
reasoning  I  are  important  in  making  these  rough  cheeks  as  well  us 
in  making  the  original  computations. 

In  wind  calculation,  particular  attention  must  be  given  to  correctly 
interpreting  the  drift  and  the  direction  of  the  wind  vector  on  the  com¬ 
puter  or  airplot.  Otherwise,  reciprocal  winds  will  be  obtained  or  the 
drift  will  be  given  in  the  wrong  direction.  Space  I  is  probably  im¬ 
portant  to  success  in  these  tasks. 

The  radar  observer  must  constantly  attend  to  his  instruments  and 
set,  keeping  truck  of  changes  in  heading,  airspeed,  ground  position, 
etc.  To  have  this  data  available  without  depending  upon  his  memory, 
he  must  record  it  in  his  log.  Diligent  attention  to  the  ever-changing 
scope  picture  will  make  his  orientational  problem  much  simpler.  A3 
previously  stated,  this  “systematic  diligence”  will  depend  upon  the 
existence  of  habits  of  doing  tusks  thoroughly,  carefully,  and  system¬ 
atically. 

Two  other  abilities  seem  associated  with  navigation  operations. 
Ono  of  these,  verbal  comprehension,  is  involved  in  learning  the  sub¬ 
ject  matter  of  navigation.  The  other,  “scientific  background”  may 
facilitate  learning  the  computational  procedures,  the  use  and  calibra¬ 
tion  of  instruments,  the  use  of  computers  and  maps,  etc. 

Bombing:  Job  Description 

Setting  up  computer  box.' — When  bombs  arc  to  be  dropped  on  a 
target  from  a  moving  aircraft,  they  must  he  dropped  at  some  point 
before  the  aircraft  is  directly  over  the  target.  Otherwise,  the  for¬ 
ward  motion  imparted  to  the  bombs  by  the  aircraft  will  carry  them 
beyond  the  target  during  the  time  of  fall.  Among  the  factors  which 
govern  how  far  ahead  of  the  target  the  bombs  should  be  released,  two 
are  of  concern  to  the  radar  observer.  These  are  tho  aircraft’s  abso¬ 
lute  altitude  and  ground  speed  on  the  bomb  run.  For  accurate  bomb¬ 
ing,  the  radar  observer  must  determine  these  factors  and  use  them 
to  prepare  the  set  for  bombing.  He  determines  the  expected  ground 
speed  and  absolute  altitude  over  the  target  before  he  reaches  the  IP 
at  the  beginning  of  the  bomb  run.  lie  then  predicts  ground  speed  by 
putting  his  latest  wind  onto  the  E-OB  and  computing  ground  speed 
on  the  true  course  from  the  IP  to  the  target.  Absolute  altitude  is 
determined  at  some  point  before  the  IP  with  tho  SCR-713  Radar 
Altimeter,  reading  in  terms  of  feet.  An  alternative  method  is  to  uso 
the  bombing  circle  on  the  AN/APS-15  or  AN/APS-15A,  setting  it 
on  the  innermost  return  on  the  PPI  scope  and  reading  the  altitndo  in 
terms  of  nautical  miles  from  the  computer  drum  on  the  AN/APS-15 
nr  in  terms  of  hundreds  of  feet  in  the  counter  window  of  tho 
AN/APS-15A.  Tho  SCR-718  reading  is  moro  accurate.  In  cither 
case  the  radar  observer  must  correct  this  altitude  for  the  difference  bo¬ 
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twcou  the  terrain  altitudes  at  (ho  target  and  at  the*  point  where  the 
absolute  altitude  was  determined. 

In  direct  bombing,  where  the  radar  observer  releases  the  bombs,  he 
sets  the  absolute  altitude  and  ground  speed  into  the  computer  box  of 
the  set.  Oil  the  AX/A  lhS-1 3  lie  adjusts  a  pair  of  cross  hairs  up  a 
nautical  mile  scale,  setting  the  horizontal  line  at  the  absolute  altitude, 
lie  then  rotates  the  computer  drum  until  the  proper  ground  sliced 
line  is  under  the  vertical  cross  luiir.  On  the  AX/APS  -lf>A,  he  twists 
a  knob  and  sets  the  absolute  altitude  on  (lie  altitude  counter  in  terms 
of  hundreds  of  feet.  lie  then  twists  another  knob  to  replace  the 
proper  ground  speed  under  the  dial  marker.  In  both  sets,  this  posi¬ 
tions  the  bright  bombing  circle  on  the  PPJ.  scope  at  a  distance  from 
the  center  such  that  when  the  target  touches  it,  the  bomb  should  be 
released. 

In  coordinated  bombing,  the  radar  set  yields  information  which  is 
put  into  the  visual  boinbsiglit  and  the  bombsight  mechanism  auto¬ 
matically  releases  the  bombs  at  the  proper  moment.  The  procedures 
carried  out  on  a  typical  coordinated  bomb  run  are  summarized  in 
figure  ‘1.(5.  Before  the  IP,  the  radar  observer  informs  the  bombadier 
of  the  altitude  and  ground  speed.  These  are  used  to  place  rate  and 
dropping  angle  into  the  bombsight.  During  the  bomb  run,  tho  radar 
observer  notifies  the  bombardier  when  the  target  is  at  certain  specified 
angles  from  the  aircraft.  These  “sighting  angles,”  measured  in  de¬ 
grees  below  the  horizontal,  are  used  to  start  the  bombsight  mechanism 
mid  to  make  adjustments  in  rate  and  dropping  angle  which  are  neces¬ 
sary  because  of  inaccuracies  in  predicted  altitude  and  ground  speed. 
For  each  angle,  the  radar  observer  positions  the  bombing  circlo  on 


3.  STARTS  KILU.3  DRIFT  OH  THE  TARCET. 
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Fiona-;  -l.fl. — Coordinated  boiub  run. 
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l]u.  pp[  scope  mid  calls  (lie  angle  when  (In?  target  touches  the  circle. 
One  of  (hi!  ml  vantages  of  coordinated  procedure  over  direct,  bombing 
i.,  that  the  radar  ob.-orver  has  a  chance  to  judge  the  coincidence  of 
bombing  circle  and  target  four  or  more  times.  Consequently  there 
0  :1  -neater  chance  that  small  errors  in  setting  lip  (In'  computer  box 
or  in  the  observing  when  (lie  target  (ouches  the  bombing  circle  will 
cancel  out. 

The  radar  observer  sets  up  the  computer  box  on  the  AN/APS-15 
for  the  fust  sighting  angle  (70°)  by  adjusting  the  cross  hairs  on  the 
nautical  mile  scale  to  the  absolute  altitude  and  rotating  the  range 
drum  to  a  point  where  the  cross  hairs  intersect  the  70°  sighting  angle 
line.  On  the  AX/APS-15A,  he  enters  tables  with  the  altitude  and 
ground  speed  and  finds  the  altitude  counter  reading  (hundreds  of 
feet)  which  lie  must  set  into  the  computer  box  for  the  70°  angle. 

All  of  the  initial  preparation  of  the  computer  box  described  above 
is  carried  out  at  the  end  of  the  navigation  leg  immediately  preceding 
the  bomb  run. 

Killing  drift. — In  order  for  the  bombs  to  fall  directly  upon  the 
targi  the  aircraft  must  be  making  a  track  for  the  target  at  the  time 
of  release.  If  the  aircraft  is  drifting  to  one  side  of  the  target,  the 
bombs  will  fall  on  that  side.  The  process  of  correcting  the  aircraft’s 
heading  until  it  is  tracking  toward  the  target  is  called  “killing  drift 
on  the  target.” 

At  tho  beginning  of  the  bomb  run,  only  the  target  area  as  a  wliolo 
is  defined  on  the  scope.  Later,  as  the  aircraft  approaches  within  5 
or  10  miles  of  the  target,  the  target  area  is  seen  as  composed  of  smaller 
sections  and  the  aiming  point  is  defined.  Consequently,  drift  is  killed 
initially  on  tho  whole  target  area  and  finally  upon  the  aiming  point. 

One  method  of  killing  drift  on  the  target  is  called  “homing.”  In 
this  procedure,  the  radar  observer  sets  on  the  target  tho  etched  lino 
which  oisccts  the  PPI  scope.  If  the  aircraft  is  tracking  toward  tho 
target,  (he  target  will  move  down  this  line  to  the  center  of  tho  scope. 
If  the  target  moves  to  the  right  of  the  line,  the  aircraft  is  drifting 
to  the  left  of  the  target  and  a  right  correction  in  heading  is  necessary. 
As  the  radar  observer  notes  in  which  direction  the  target  moves,  ho 
gives  small  arbitrary  corrections  to  the  pilot.  lie  continues  this  trial- 
and-ermr  procedure  until  tho  target  is  moving  directly  down  tho 
line. 

A  more  accurate  drift-killing  procedure  is  tho  “multiple-drift 
method.”  The  radar  observer  reads  the.  bearing  of  tho  target  before 
and  after  it  has  moved  one-fifth,  one- fourth,  or  one-third  of  tho  way 
toward  the  center  of  the  scope.  lie  multiplies  tho  difference  between 
tho  two  bearings  by  5,  I,  or  3,  tho  recipiocal  of  the  fraction  used,  and 
gives  this  number  of  degrees  as  a  correction  to  tho  pilot.  Again  ho 
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must  give  (ho  correction  to  the  right  if  the  target  moved  to  the  right 
of  the  fir.'t  hearing  < >r  to  the  left  if  it  Ini'  moved  left. 

The  radar  oh'crver  may  me  the  range  marks  to  delineate  tho  por¬ 
tion  of  the  di.'tanoe  the  target  moves.  For  example,  he  can  follow  the 
target  as  it  moves  from  the  20-  to  the  la-mile  range  marks,  in  which 
cn.'O  the  target  would  have  moved  one-fonrth  of  the  distance  toward 
the  center.  Some  sets  have  lines  etched  on  the  face  of  the  scope  mark¬ 
ing  off  one-fonrth  or  one-fifth  of  the  distance  to  the  center.  To  uso 
these  most  efficiently,  the  radar  observer  manipulates  a  control  to 
move  the  target  out  to  the  outer  mark  when  beginning  a  multiple-drift 
reading. 

Killing  rale  and  bomb  release  procedure. — As  the  aircraft  moves 
closer  to  the  target,  the  radar  observer  manipulates  controls  to  reduce 
tho  area  of  terrain  pictured  on  tho  scope.  This  increases  tho  size  of 
terrain  features  and  finally  results  in  definition  of  the  various  sections 
of  the  target  area.  He  also  adjusts  controls  concerned  with  tuning  tho 
set  to  improve  the  scope  picture.  Radar  observers  frequently  turn  on 
“sector  scan”  by  means  of  which  only  a  portion  of  the  terrain  below 
the  aircraft  is  presented  on  tho  scope.  The  sweep  covers  this  sector 
much  more  frequently  than  it  does  when  traversing  the  entire  scope, 
thus  facilitating  examination  of  tho  target.  When  tho  aiming  point 
has  been  identified,  the  radar  observer  gives  small  arbitrary  correc¬ 
tions  to  kill  any  residual  drift. 

In  direct  bombing,  the  radar  observer  switches  on  tho  bombing 
circle  early  in  tho  bomb  run.  Tho  aiming  point  becomes  visible  and 
tracks  toward  tho  aircraft.  The  radar  observer  toggles  tho  bomb 
release  switch  at  tho  exact  moment  he  secs  the  near  edge  of  tho  aiming 
point  touch  the  outer  edge  of  the  bombing  circle. 

In  coordinated  bombing,  when  tho  target  approaches  the  bombing 
circle  positioned  for  the  TO0  sighting  angle,  the  radar  observer  requests 
the  crew  to  stay  off  tho  interphone  during  tho  bomb  run,  and  warns 
the  bombardier  that  the  70°  sighting  angle  is  coming  up.  When  the 
near  edge  of  the  aiming  point  touches  the  bombing  circle,  tho  radar 
observer  calls,  “Mark.”  lie  quickly  and  accurately  resets  tho  com¬ 
puter  box  for  the  next  sighting  angle.  As  the  aiming  point  approaches 
the  circle  a  second  time,  he  again  warns  the  bombardier  and  specifies 
tho  angle.  When  the  aiming  point  touches  tho  bombing  circle,  he 
again  calls,  “Mark.”  lie  resets  the  computer  box  and  repeats  the 
procedure  for  as  many  angles  as  possible.  A  radar  observer  usually 
calls  four  or  five  angles  on  a  single  coordinated  run.  Meanwhile,  ho 
manipulates  the  controls  of  the  set  to  reduce  the  area  presented  on  the 
scope  and  to  increase  the  definition  of  the  aiming  point.  As  tho  aiming 
point  becomes  defined,  ho  usually  has  to  givo  small  last-minute  drift 
corrections.  Tho  process  of  killing  rate  and  drift  on  a  coordinated 
bomb  run  is  summarized  in  figure  4.6. 


The  release  point  eonc-pouds  to  :i  .'it)0  sighting  angle  for  a  ground 
hj)u<I  of  ISO  miles  per  limn-  and  an  altitude  of  approximately 
10  OUO  feet.  'I  he  coordinated  proeednre  from  the,  70°  angle  to  the 
release  point  is  carried  out  over  a  distance  of  1  to  5  miles  or  a  time 
of  approximately  P/_.  minutes.  The  entile  homb  run  on  training 
mT-ioiis  is  about  30  miles  long,  which  allows  approximately  10  min¬ 
utes  for  the  drift  killing  and  release  proeednre. 

Bombing:  Job  Analysis 

On  the  bomb  run,  the  radar  observer  has  much  to  do  in  n  short 
period  of  time.  It  is  essential  that  he  plan  the  flight  front  before  the 
IP  to  ‘‘Bombs  Away.”  This  plan  should  designate  specific  points  at 
which  to  do  nil  essential  operations,  such  as  obtaining  absolute  alti¬ 
tude,  computing  ground  speed,  preparing  the  computer  box,  and 
linking  multiple-drift  corrections.  Planning  is  especially  important 
in  resetting  the  computer  box  foi  successive  sighting  angles  on  a  coor¬ 
dinated  run.  This  process  must  be  done  quickly  and  accurately. 
Systematic  planning  of  this  sort,  in  all  probability,  is  related  to 
reasoning  I  (general  reasoning),  “scientific  background,”  and  “sys¬ 
tematic  diligence.” 

Setting  and  resetting  the  computer  box  also  involves  “immediate 
memory  span,”  psychomotor  precision,  and  scale  and  table-reading 
ability.  Scale  reading  is  involved  in  obtaining  absolute  altitude,  com¬ 
puting  ground  speed,  and  setting  the  values  on  the  computer  scales. 
As  previously  stated,  success  in  scale  reading  may  be  dependent  upon 
numerical  facility,  space  I  (spatial  relations),  and  “scientific  back¬ 
ground.” 

To  kill  drift,  the  nular  observer  must  first  select  the  proper  section 
of  the  target  area  and  later,  when  it  becomes  visible,  identify  the  aim¬ 
ing  point.  The  requirements  involved  here  appear  to  he.  the  same  as 
those  of  scope  interpretation  described  above,  namely,  perceptual 
speed,  memory  II  (visual  memory),  and  space  I.  The  abilities  re¬ 
quired  for  interpreting  direction  of  drift,  memory  II  and  space  I,  have 
also  been  discussed  previously. 

For  the  multiple  drift  procedure  to  bo  accurate,  each  of  the  two 
hearings  must  be  taken  on  the  same  part  of  tiro  target.  When  tho 
target  has  moved  toward  the  center  of  the  scope  the  necessary  distance, 
the  radar  observer  must  remember  and  use  exactly  tire  same  section  of 
the  target  that  ho  used  in  reading  the  first  bearing.  This  probably  re¬ 
quires  memory  EC. 

On  the  bomb  run,  tho  radar  observer  is  faced  with  a  complex  task 
that  must  bo  performed  quickly  and  accurately.  He  must  go  through 
the  multiple  drift  procedure,  continuously  give  refining  correction:.', 
continuously  expand  the  picture  on  tire  scope,  manipulate  the  gain  and 
tilt  controls  to  improve  tho  picture,  givo  the  bombardier  the  latest 
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ground  speed  and  drift  information,  warn  the  bombardier,  call  the 
tirst  sighting  angle,  reset  the  computer  box  and  call  three  or  four 
further  sighting  angles,  set  the  computer  box  for  direct  drop,  make 
final  drift  corrections,  and  be  prepared  to  release  the  bombs  if  neces¬ 
sary.  The  complexity  and  stress  of  this  task  indicate  that  “emotional 
control”  or  resistance  to  confusion  are  important  in  radar  observer 
success. 

Summary 

Thus  far  in  the  job  analysis,  no  direct  statements  have  been  made  as 
to  the  relative  importance  of  the  various  abilities  or  factors  which 
have  been  described.  In  the  following  list,  these  abilities  are  grouped 
according  to  their  estimated  validity  for  predicting  over-all  success  in 
radar  observer  training.  A  more  quantitative  prediction  is  not  made 
since  the  reliability  of  the  criterion  will  affect  the  absolute  size  of  the 
validity  coefficients.  A  summary  will  state  the  aspects  of  the  radar 
observer’s  job  for  which  each  ability  is  thought  to  be  important. 

Abilities  predicted  to  have  highest  validity. — “Scientific  back¬ 
ground”  appears  to  be  important  in  developing  a  rationale  for  the 
operating  procedures,  remembering  the  appearance  of  returns  from 
various  terrain  features,  and  learning  and  planning  the  navigation 
work. 

Space  I  appears  to  be  important  in  interpreting  the  motion  of  re¬ 
turns  across  the  scope,  using  the  correct  wind  and  drift  direction, 
and  reading  dials. 

Memory  II  is  important  in  tuning  the  set,  identifying  target  areas 
and  aiming  points,  perceiving  the  motion  of  returns  across  the  scope, 
and  taking  the  two  multiple  drift  bearings  on  the  same  part  of  the 
target. 

Abilities  predicted  to  have  relatively  high  validity. — “Systematic 
diligence”  is  probably  important  in  organizing  and  correctly  carrying 
out  operational  procedures,  organizing  the  navigational  work  to  pro¬ 
vide  maximum  time  for  every  operation,  keeping  track  of  flight  data, 
keeping  an  adequate  log  of  navigational  and  flight  data,  and  planning 
the  bomb  run. 

Numerical  facility  is  believed  important  in  doing  computations  and 
making  computational  checks  and  in  reading  dials,  scales,  and  tables. 

Rea  toning  I  is  probably  important  in  planning  the  navigation  log 
and  bomb  run  procedures,  making  computations,  and  checking  them. 

Perceptual  speed  is  probably  related  to  identifying  individual  re¬ 
turns  and  patterns  of  returns. 

Verbal  comprehension  would  seem  to  be  essential  primarily  at  the 
learning  stages  in  acquiring  technical  information. 

Abilities  predicted  to  have  relatively  law  validity. — Length  esti¬ 
mation,  together  with  size  and  angle  estimation,  is  probably  involved 
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in  remembering  and  identifying  patterns  of  returns  and  in  scale,  dial, 
and  table  reading. 

,  Fsychomotor  precision  is  probably  important  in  manipulating  the 
switches,  knobs,  and  screw-driver  adjustments  on  the  set,  handling 
the  Weems  plotter,  E-GB  computer,  and  dividers,  and  setting  and 
resetting  the  computer  box  during  the  bomb  run. 

Mechanical  experience  may  facilitate  the  development  of  a  rat  ionalo 
for  operating  procedures. 

Abilities  predicted  to  have  lowest  validity. — Memory  l  is  involved 
in  learning  and  remembering  the  procedures  for  starting,  tuning,  and 
calibration. 

“Visual  acuity”  and  “brightness  discrimination”  are  involved  in 
tuning  and  perceiving  patterns  of  returns.  Individual  differences  nro 
probably  not  great  among  radar  observer  students  in  these  variables. 

“Emotional  control”  is  probably  important  in  carrying  out  the  com¬ 
plex  task  on  the  bomb  run  rapidly  and  accurately.  Valid  measures  of 
emotional  control  are  not  known. 

Abilities  predicted  to  have  no  validity. — Abilities  among  those  de¬ 
fined  at  the  beginning  of  the  chapter  which  probably  have  no  validity 
are::  Psychomotor  coordination,  visualization,  space  il  or  rotational 
space,  and  pilot  interest. 

JOB  DESCRIPTION  OF  TIIE  RADAR  OBSERVER  IN  COMBAT 

Sources  of  Information 

This  sketch  of  the  radar  observer’s  job  in  combat  is  based  upon  the 
following  sources  of  information:  (a)  Personal  accounts  of  radar  ob¬ 
servers  from  the  Eighth  Air  Force,  (b)  outlines  of  Standard  Operating 
Procedures  for  radar  navigation  and  bombing  from  the  Twentieth 
and  Twenty-first  Bomber  Commands  of  the  Twentieth  Air  Force,  (c) 
operational  analyses  from  the  Eighth,  Fifteenth,  and  Twentieth  Air 
Forces,  and  ( d )  articles  in  tlw  jnagazino  “Radar”  which  describe  the 
use  of  radar  equipment  in  the  various  theaters.7  It  must  he  noted 
that  variability  in  tasks  performed  by  the  radar  observer  within  and 
between  the  various  units  in  a  given  theater  is  much  greater  than  in 
training.  An  attempt  is  made  below  to  describe  the  performance  of 
a  representative  radar  observer  in  each  theater.  Space  does  not  permit 
summarizing  all  the  known  improvisations  and  variations. 

European  Theater 

Equipment. — The  B-17  and  B-21  aircraft  used  in  Europe  were 
equipped  with  AN/APS-15  and  AN/APS-15A  radar  sets.  The 
operation  of  these  sets  was  identical  with  that  described  in  tho  job 
description  of  set  operation  in  training. 

7  Rndiir :  No.  7.  1  Jan.  1045;  No.  0,  30  Ajirll  1015;  No.  10,  30  Juno  1045. 
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Xaviyaiion. — Early  radar  observers  were  rated  navigators  who  had 
littlo  initial  knowledge  of  the  radar  set.  Only  the  lead  ships  were 
equipped  with  radar,  there  being  approximately  one  such  ship  for 
each  squadron  of  12  planes.  In  the  lead  ship  were  two  other  navi¬ 
gators,  the  lead  navigator  who  did  pilotage  navigation  and  a  second 
navigator  responsible  for  DR  or  dead  reckoning  navigation.  The 
isular  observer  operated  the  set,  gave  fix  information  to  the  DR 
navigator  every  few  minutes,  and  occasionally  computed  winds  to 
servo  us  checks  on  the  DR  navigator.  The  airplot  method  of  deter¬ 
mining  winds  was  used  for  the  most  part.  The  radar  observer’s 
primary  responsibility  in  navigation  was  to  lead  the  formation 
around  flak  areas  when  such  areas  were  visible  on  the  scope.  This 
was  carried  out  by  radar  pilotage,  the  purpose  being  to  avoid  such 
areas  by  at  least  10  miles.  The  radar  observer  simply  gave  headings 
to  keep  the  returns  from  the  identified  flak  areas  beyond  the  10-mile 
range  mark  on  the  PPI  scope. 

Identification  of  check  points  for  use  in  navigation  was  fairly  easy 
over  the  coastal  areas.  It  was  more  complicated  over  central  Ger¬ 
many  where  identification  had  to  be  made  largely  on  the  basis  of 
returns  from  towns.  Check  point  identification  was  particularly 
difficult  in  the  mountainous  terrain  of  southern  Europe  covered  by 

the-  Fifteenth  Air  Force  operating  from  Italy. 

% 

Bombing. — The  DR  navigator  was  responsible  for  bringing  the 
formation  to  the  IP.  The  radar  observer  usually  gave  the  heading 
to  the  target  from  the  IP  and  made  the  initial  drift  corrections  on  the 
bomb  run.  The  range  of  the  set  allowed  him  to  pick  up  the  target 
long  before  it  could  be  detected  visually.  This  advantage  permitted 
him  to  give  more  accurate  headings  and  initial  drift  corrections  than 
the  other  navigators. 

The  radar  observer  set  up  his  equipment  for  bombing  regardless 
of  weather  conditions.  ).i  there  was  an  undercast,  lie  controlled  the 
bomb  release.  Whether  or  not  visual  conditions  prevailed,  lie  pre¬ 
pared  to  do  direct  radar  bombing,  always  being  ready  to  take  over 
if  the  bonibadier  could  not  make  the  release  visually.  Up  to  shortly 
before  D-day,  all  radar  bombing  was  direct  bombing.  After  its  in¬ 
troduction,  coordinated  bombing  rapidly  became  the  preferred  pro¬ 
cedure. 

Identification  of  targets  and  aiming  points  proved  to  bo  a  major 
difficulty.  Most  targets  were  well  inland  so  that  identification  had 
to  be  made  without  aid  from  coastal  returns.  The  sets  provided  poor 
definition  of  returns  compared  with  later  sets  used  in  the  Pacific 
Theater.  Identifying  the  aiming  point  usually  involved  estimating 
its  position  within  a  large  homogeneous  return.  In  the  early  stages  of 
operations  with  radar  the  quantity  and  quality  of  scope  photos  for 
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briefing  was  not  adequate  to  familiarize  radar  observers  with  returns 
from  the  target  area. 

Pacific  Theater 

Equipment. — ■'While  B-24’s  and  B-17?s  with  AN/APS-15  and 
AN/APS-15A  sets  were  used  in  tho  Pacific  for  sea  search,  mine-lay¬ 
ing,  and  regular  bombing  missions,  the  most  frequent  airborne  radar 
installation  in  this  theater  was  tho  AN/APQ-13  set  in  13-29  aircraft. 
A  few  AN/APQ-7  sets  in  B-29  aircraft  wero  used.  Tho  AN/ 
APQ-13,  shown  in  a  B-29  installation  in  figure  4.7,  is  comparable  to 
the  AN/APS-15  in  function,  controls,  and  operation,  but  provides 


slightly  greater  definition  of  returns.  Tho  AN/APQ-7,  a  new  de¬ 
velopment,  is  characterized  by  much  greater  definition  on  tho  PPI 
scopo  but  the  presentation  is  limited  to  an  area  of  only  GO0  wide  and 
directly  ahead  of  tho  aircraft. 

Navigation. — Airborno  radar  sets  in  tho  Pacific  wero  first  operated 
by  specially  trained  enlisted  aircrew  members,  designated  as  radar 
operators.  Because  they  know  little  about  navigation,  this  phaso  of 
their  work  was  supervised  by  tho  navigator.  The  radar  operator 
simply  controlled  tho  set  and  made  adjustments  as  directed  by  tho 
navigator  on  tho  basis  of  returns  on  tho  auxiliary  PPI  scopo.  Tho 
navigator  took  fixes  from  tho  auxiliary  scopo  and  computed  target¬ 
timing  winds  as  described  on  page  39.  Aiming  point  identification 
was  carried  out  in  cooperation  with  the  navigator.  Tho  radar  operator 
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set  un  the  bombing  problem  with  information  obtained  from  the 
bombardier.  The  navigator  hilled  drift  on  the  bomb  run  and  either 
the  navigator  or  radar  operator  called  the  successive  sighting  angles 
(coordinated  bombing  procedure),  the  radar  operator  resetting  the 
computer  box  each  time. 

'Idle  enlisted  radar  operator  was  gradually  replaced  by  the  com¬ 
missioned  radar  observer.  As  a  result  of  bis  specialized  training, 
the  radar  observer  was  able  to  assume  more  navigational  and  bombing 
duties.  Besides  operating  the  sot,  the  radar  observer  took  fixes  at 
regular  intervals,  relayed  the  information  to  tho  navigator,  computed 
winds  from  radar  information,  and  gave  them  to  the  navigator  as 
checks.  Winds  wore  determined  largely  by  the  target-timing  me  tho  , 
with  the  target  being  tracked  for  6  to  10  minutes. 

For  a  large  part  of  the  over-water  flight  to  Japan,  no  radar  returns 
were  available  and  navigation  was  based  upon  Loran,  driftmoter, 
and  celestial  data.  However,  because  of  the  long  range  of  the  radar 
set,  the  radar  observer  was  able  to  pick  up  the  Japanese  coast  in  time 
to  aid  the  navigator  in  crossing  the  coastline  at  the  briefed  point.  Ac¬ 
curacy  of  this  first  contact  was  vital  to  accuracy  of  the  approach  to 
the  IP  and  turn  onto  the  bomb  run.  The  radar  observer  gave  in¬ 
formation  as  to  when  the  formation  should  start  to  turn  over  the  IP 
in  order  to  corne  out  of  the  turn  onjdie  briefed  course.  The  high 
velocity  winds  encountered  over  Japan  made  it  necessary  to  carry  out 
all  such  navigational  procedures  quickly  and  accurately.  All  radar 
information  was  carefully  checked  against  data  from  other  sources. 

The  Japanese  islands  gave  excellent  radar  returns.  Once  tho  for¬ 
mation  came  within  100  miles  of  the  island  chain,  numerous  naviga¬ 
tional  check  points  were  available.  The  identification  problem  ap¬ 
peared  simple  and  there  was  a  tendency  to  abandon  DR  navigation 
and  rely  wholly  upon  radar  pilotage.  However,  experience  showed 
that  identification  of  returns  over  Japan  was  not  as  easy  as  it  first 
seemed.  Many  islands  and  inlets  looked  alike  and  the  patterns  made 
by  the  land  and  water  returns  were  extremely  complex.  Missions 
were  most  successful  when  there  was  close  cooperation  between  the 
DR  navigator  and  the  radar  observer,  each  checking  the  other. 

On  the  return  legs  of  the  mission,  few  radar  returns  were  available, 
Under  maximum  range  conditions,  distant  island  chains  were  identi¬ 
fiable  and  finally  the  home  islands  provided  fix  information,  Because 
of  the  paucity  of  navigational  data  on  these  missions,  the  radar  ob¬ 
server  bad  to  maintain  the  set  at  its  maximum  performance  to  obtain 
all  possible  position  data  for  tho  navigator. 

Bombing. — Tho  navigator  was  responsible  for  directing  the  plane 
to  the  IP.  Under  nonvisual  conditions,  the  radar  observer  was  re¬ 
sponsible  for  identifying  the  aiming  point,  making  course  corrections, 
arid  calling  sighting  angles  on  the  coordinated  bomb  run.  Because  of 
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the  large  size  and  homogeneity  of  returns  irom  the  cities  and  built-up 
areas  of  Japan,  the  visual  aiming  point  of  the  primary  target  often 
was  not  identifiable  on  the  PPI  scope.  In  such  instances,  under  non¬ 
visual  conditions,  a  secondary  target  which  yielded  characteristic  and 
identifiable  radar  returns  was  used.  However,  since  a  sizable  part 
of  bombing  over  Japan  had  to  be  carried  out  by  radar,  it  was  impossible 
strategically  to  continue  to  neglect  the  primary  target  and  bomb  only 
the  radar  targets.  Consequently,  offset  bombing  systems  were  de¬ 
veloped  for  bombing  such  targets  by  radar  even  though  they  wero  not 
visible  on  the  PPI  scope.  These  systems  depend  upon  the  use  of  an 
identifiable  aiming  point  at  a  known  distance  and  bearing  from  the 
target.  Offset  bombing  requires  extremely  accurate  headings  from 
the  IP  to  target  and  either  timing  the  High t  from  the  aiming  point  to 
the  target  or  making  already-computed  adjustments  in  slant  rango 
settings.  Because  of  these  difficulties,  offset  methods  were  never  com¬ 
pletely  satisfactory. 

In  general,  in  spite  of  the  greater  definition  yielded  by  the 
AX/APQ-13  and  AX/APQ-7,  the  radar  observer  usually  has  to  esti¬ 
mate  the  position  of  the  aiming  point  within  a  large  homogeneous 
target  return. 


JOB  ANALYSIS  OF  THE  RADAR  OBSERVER  IN  COMBAT 


The  information  contained  in  the  foregoing  job  description,  for  tho 
most  part,  lacks  sufficient  detail  to  make  possible  a  thorough  analysis 
of  the  radar  observer’s  job  in  combat.  However,  general  estimates  can 
be  made  of  the  importance  to  combat  success  of  tho  abilities  judged  to 
be  necessary  for  success  in  training.  While  these  estimates  are  based 
upon  all  evidence  available,  they  are  advanced  with  full  recognition  of 
their  tentative  nature.  No  new  abilities  are  introduced;  the  evidence 
at  hand  seems  inadequate  to  justify  this  step. 


Set  Operation 

Set  operation  formed  a  relatively  larger  part  of  the  radar  observer’s 
task  in  combat  than  in  training  since  he  had  fewer  other  responsibilities 
and  was  continually  called  upon  to  furnish  position  information.  On 
the  other  hand,  it  is  probable  that  skill  of  this  sort  had  become  semi¬ 
automatic  through  continued  practice  by  the  time  tho  observer  readied 
his  overseas  station.  The  correct  ion  of  set  mal  functions  and  set  opera¬ 
tion  under  atypical  weather  condition,  however,  became  correspond¬ 
ingly  more  important.  It  seems  likely  that  these  changes  tended 
to  deemphnsize  t lie  importance  of  memory  I  (rote  memory),  verbal 
comprehension,  and  psychomotor  precision  and  increase  emphasis 
upon  mechanical  experience  and  “scientific  background.” 
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Scope  Interpretation 

Identification  of  returns  was  difficult  in  all  theaters.  However,  the 
the  Dll  navigator  afforded  a  check  upon  the  radar  observer  in  this  re¬ 
spect  since  he  plotted  the  radar  fixes,  computed  successive  ground  speed 
and  tracks,  and  had  the  radar  observer  retake  fixes  which  were  incon¬ 
sistent  with  previous  position  data.  In  general,  it  appears  that  scope 
interpretation  was  equally  as  important  and  required  the  same  abilities 
in  combat  as  in  training.  It  will  be  remembered  that  the  factors 
thought  to  be  most  important  for  mastering  scope  interpretation  in 
training  were  perceptual  speed,  memory  II  (visual  memory),  and 
space  I  (spatial  relations). 

Navigation 

The  radar  observer  shared  his  navigation  tasks  with  the  DR  navi¬ 
gator  so  that  his  work  on  navigation  legs  of  missions  was  limited 
mostly  to  set  operation  and  fix-taking.  In  this  respect,  combat  mis¬ 
sions  probably  required  less  planning  of  activities,  which  is  thought  to 
entail  “systematic  diligence,”  reasoning  I  (general  reasoning),  and 
“scientific  background,”  and  less  computational  work,  thought  to  ve 
quire  numerical  facility  and  reasoning  I  and  little  wind  calculation 
thought  to  require  space  I.  It  is  likely  that  individual  differences  m 
reading  dials,  scales,  and  the  range  marks  on  the  PPI  scope  were  less 
marked  among  radar  observers  in  combat  than  in  training.  This 
would  act  to  decrease  the  relative  importance  of  numerical  facility, 
space  I,  and  length  estimation.  Verbal  comprehension  should  also 
prove  to  be  less  important  in  combat  than  in  training  since  it  was  in- 
involved  primarily  in  learning  the  navigational  procedures. 

Bombing 

Aside  from  ihe  emotional  stress,  the  combat  bomb  run  was  closely 
simulated  by  training  bomb  runs.  A  relatively  unimportant  difference 
was  that  in  combat,  the  radar  observer  often  obtained  necessary  ground 
speed  and  altitude  information  from  other  crew  members  without 
having  to  determine  it  himself.  This  may  have  decreased  the  com¬ 
plexity  of  bombing  duties  required  of  the  radar  observer  and  hence 
simplified  his  planning  problem.  This  would  decrease  the  importance 
of  reasoning  I,  “scientific  background,”  and  “systematic  diligence” 
which  uro  thought  to  bo  important  in  organization  of  duties.  As  in 
the  caso  of  navigation,  we  might  expect  overlearning  to  have  mini¬ 
mized  individual  differences  in  reading  dials  and  scales  during  the  com¬ 
bat  bomb  run.  However,  the  increased  stress  of  the  bomb  run  may 
have  caused  these  differences  to  bo  reemphasized.  The  factors  in¬ 
volved  in  scale  reading,  it  is  believed,  are  numerical  facility,  “scien¬ 
tific  background,”  and  space  I.  Identification  of  aiming  points  for 
killing  drift  and  rate  was  probably  more  difficult  in  combat  than 
training,  although  oven  in  training  it  was  customary  to  bomb  un- 
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familiar  targets.  This  task  is  thought  to  require  perceptual  speed, 
memory  II,  and  space  I. 

Summary 

In  summary,  it  appears  that  the  factors  involved  in  the  correction 
of  set  malfunctions  and  in  operating  the  set  under  typical  conditions, 
namely  mechanical  experience  and  “scientific  background”  and  in  in¬ 
terpreting  the  scope,  perceptual  speed,  memory  II  (visual  memory), 
space  I  (spatial  relations),  represent  the  most  important  abilities  re¬ 
quired  of  the  successful  radar  observer  in  combat.  Factors  involved 
primarily  in  carrying  out  navigational  computations,  numerical  fa¬ 
cility  and  reasoning  I  (general  reasoning),  and  in  organizing. the 
separate  duties  on  the  navigation  legs  and  bomb  runs,  “systematic 
diligence,”  and  reasoning  I,  probably  become  less  important  in  com¬ 
bat  than  in  training.  Factors  involved  primarily  in  learning  set  op¬ 
eration,  navigation,  and  bombing  procedures,  such  as  verbal  compre¬ 
hension,  and  memory  I  (rote  memory)  arc  thought  to  become  relatively 
unimportant  in  combat. 

It  has  often  been  pointed  out  that  aircrew  members  were  under  great 
emotional  stress  in  combat.  Although  no  adequate  test  has  been 
found  to  measure  emotional  control,  it  is  undoubtedly  true  that  it  is 
more  important  in  combat  than  in  training. 

TIIE  JOB  OF  TIIE  RADAR  OBSERVER  IN  TIIE  FUTURE 

Trends  in  Equipment  Development 

The  recent  developments  in  airborne  radar  equipment  indicate 
probable  changes  in  the  tasks  required  of  the  radar  observer.  The 
most  important  of  the  equipment  trends  ns  illustrated  by  tho 
AN/APQ-13  and  later  sets  are  briefly  as  follows : 

(1)  Greatly  increased  definition  which  means  that  terrain  features 
will  be  presented  in  greater  detail  on  the  PPI  scopo. 

(2)  Automatic  solution  of  navigation  problems  by  electronic  com¬ 
puters  integrated  with  the  radar  set.  These  computers  will  yield  such 
information  as  track,  ground  speed,  and  drift. 

(3)  Computers  permitting  extensive  use  of  offset  bombing.  These 
computers  will  make  adjustments  automatically  in  the  bombing  vari- 
abl  cs  to  permit  aiming  on  any  identifiable  return  at  a  known  distance 
and  bearing  from  the  target  without  restricting  the  direction  of  ap¬ 
proach  to  tho  target. 

(!)  Auxiliary  systems  which  replace  the  bomb-sight  in  coordinated 
bombing  or  which  provide  for  mechanically  synchronizing  tho  bomb- 
sight  with  the  movement  of  returns  across  the  PPI  scope.  By  means 
of  these  units,  tho  radar  observer  can  kill  rate,  now  accomplished  by 
calling  successive  sighting  angles,  and  drift  by  making  adjustments 
to  Keep  the  target  under  appropriate  markers  on  tho  PPI  scope.  Other 
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bombing  aids  include  automatic  sweep  expansion  by  means  of  which 
returns  are  kept  near  the  edge  of  the  scope  and  automatic  sector  scan 
which  reduces  the  presentation  on  the  PPI  scope  to  a  sector  which  in¬ 
cludes  the  target  area. 

Indicated  Changes  in  the  Radar  Observer’s  Job 

It  is  not  clear  what  cfTect  the  increased  definition  obtained  by  new 
radar  equipment  will  have  upon  the  task  of  scope  interpretation. 
Because  more  details  of  the  terrain  are  presented  on  the  PPI  scope, 
it  would  seem  that  the  problem  of  identifying  check  points,  target 
ureas,  and  aiming  points  will  be  somewhat  simplified.  However, 
experience  in  the  Pacific  theater  with  the  AN/APQ-7  does  not  con¬ 
firm  this.  While  increased  definition  yields  a  picture  which  ap¬ 
proaches  an  aerial  photograph  in  terms  of  complexity  or  number  of 
elements,  the  increase  in  complexity  is  not  accompanied  by  an  in¬ 
crease  in  the  number  and  kind  of  cues  that  are  available  for  distinguish¬ 
ing  the  elements  from  each  other.  The  resulting  picture,  over  most 
target  areas,  consists  of  an  extremely  complicated  pattern  of  returns 
that  differ  from  each  other  only  slightly  in  terms  of  size,  shape,  and 
brightness. 

The  bombing  identification  problem  will  undoubtedly  be  simplified 
by  the  availability  of  simple  ofTset  bombing  methods.  The  radar 
observer  will  not  have  to  detect  the  aiming  point  or  estimate  its  posi¬ 
tion  in  the  target  area.  Instead  he  will  be  able  to  select  any  nearby 
sharp  return  which  lie  can  identify  on  his  map,  set  up  the  automatic 
otlset  bombing  computer,  and  aim  on  the  sharp  return.  In  general, 
however,  it  seems  likely  that  the  skills  required  in  scope  interpretation 
will  retain  their  importance  for  the  future  radar  observer.  Automatic 
solution  of  navigation  problems  will  greatly  reduce  the  computational 
skills  required  of  the  radar  observer.  lie  will  simply  have  to  manip¬ 
ulate  the  controls  on  the  computers  and  radar  set,  tracking  returns 
across  the  scope.  Use  of  the  E-GB  computer  will  be  greatly  reduced. 
Map-plotting  will  be  reduced  and  simplified.  Less  over-all  under-, 
standing  of  the  relationships  between  the  various  navigational  vari¬ 
ables  will  be  necessary.  However,  the  addition  of  computers  and 
auxiliary  units  to  the  basic  air-borne  radar  set  will  increase  the  num¬ 
ber  of  controls  which  the  radar  observer  will  have  to  use  and  the 
number  of  dials  he  will  have  to  read  and  interpret. 

In  summary,  it  seems  likely  that  the  skills  required  in  carrying  out 
present  radar  navigation  duties  will  decrease  in  importance  and  that 
skills  similar  to  or  identical  with  those  involved  in  present  set  oper¬ 
ation  or  radar  bombing  procedure  will  become  more  essential. 

SUMMARY 

The  chapter  is  introduced  by  a  discussion  of  the  purpose  and  meth¬ 
ods  of  describing  a  job.  The  distinction  is  made  between  a  “job 
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description,”  which  is  a  description  in  terms  specific  to  the.  job,  and 
a  l,job  analysis”  or  description  in  terms,  of  general  abilities. 

Following  a  brief  introduction  to  airborne  radar  equipment,  the 
job  of  the  radar  observer  in  training  is  described  under  four  head¬ 
ings:  set  operation,  scope  interpretation,  navigation,  and  bombing. 
Each  section  is  presented  as  a  job  description  followed  by  a  job 
analysis.  The  job  analysis  is  made  primarily  in  terms  of  factors 
isolated  in  factor  analyses  carried  out  in  the  Aviation  Psychology 
Program.  Where  these  factors  arc  inadequate  to  account  for  the 
skills  presented  in  the  job  description  hypothetical  abilities  are  called 
upon. 

Tho  description  of  the  radar  observer’s  job  in  training  is  concluded 
by  a  summary  of  the  job  analysis  sections.  In  this  summary  est  imates 
are  made  of  the  relative  validities  of  the  various  factors  for  predicting 
success  in  radar  observer  training.  Factors  predicted  to  have  tho 
highest  validity  were  “scientific  background,”  space  I  (spatial  rela¬ 
tions),  and  memory  II  (visual  memory).  Those  predicted  to  have 
relatively  high  validity  were  “systematic  diligence,”  numerical 
facility,  reasoning  I  (general  reasoning),  perceptual  speed,  and  verbal 
comprehension.  Factors  estimated  to  have  low  validity  were  length 
estimation,  psychomotor  precision,  and  mechanical  experience. 

The  job  of  the  radar  observer  in  combat  is  described  for  the  European 
and  Pacific  Theaters  of  Operation.  The  combat  job  is  briefly  analyzed 
by  comparing  it  with  the  analysis  of  tasks  required  in  training.  It 
is  predicted  that  the  following  factors  would  have  relatively  higher 
validity  for  predicting  success  in  combat  than  in  training:  mechanical 
experience,  “scientific  background,”  perceptual  speed,  memory  II,  and 
space  I.  Numerical  facility,  reasoning  I,  “systematic  diligence,”  and 
verbal  comprehension  arc  estimated  to  have  relatively  lower  validity 
for  combat  than  for  training. 

A  final  section  of  the  chapter  summarizes  recent  trends  in  tho  de¬ 
velopment  of  airborne  radar  equipment.  These  trends  are  inter¬ 
preted  as  indicating  that,  in  the  future,  skills  involved  in  present 
radar  navigational  tasks  will  become  less  important,  while  skills  im¬ 
portant  to  present  s:et  operation  and  radar  bombing  will  become  more 
important. 
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CHAPTER  FIVE _ 

The  Development  of  Printed 
Proficiency  Tests  1 

■  I 


INTRODUCTION 

The  Radar  Project  provided  the  radar  observer  schools  with  bat¬ 
teries  of  standardized  proficiency  measures  including  both  printed 
tests  and  performance  checks.  It  is  the  purpose  of  this  chapter  to 
describe  the  development  of  the  printed  tests.  The  chapter  which 
follows  this  one  will  describe  the  development  of  the  performance 
checks. 

Before  discussing  the  specific  use  of  proficiency  tests  in  the  radar 
observer  training  program,  the  functions  served  by  standardized 
measurement  in  such  a  program  may  bo  outlined  briefly.  Measure¬ 
ment  under  standard  conditions  yields,  of  course,  more  reliable  results 
than  instructor  ratings  of  students  and  informal  classroom  quizzes. 
Tho  benefits  resulting  from  this  greater  reliability  are  many  and 
warrant  tho  effort  required  to  initiate  and  supervise  the  administra¬ 
tion  of  a  standardized  proficiency  measurement  program.  Most  im¬ 
portant  of  these  benefits  is  tho  possibility  of  maintaining  constant  and 
uniform  standards  for  graduation  from  training.  Tho  reliable 
grades  assigned  students  at  graduation  have  the  further  effect  of 
making  possible  more  clfectivo  selection  for  subsequent  operational 
assignments.  In  addition,  such  grades  servo  ns  a  moro  reliable  cri¬ 
terion  against  which  to  validate  selection  tests.  Finally,  reliable 
course  grades  constitute  a  criterion  for  use  in  studies  of  instructor 
effectiveness,  training  methods  and  devices,  nnd  student  differences 
and  weaknesses.  Reliable  grades  assigned  at  intermediate  stages  in 
the  training  provide  valuable  information  about  student  progress. 
One  incidental  adva.  tage  of  the  standardized  measurement  program 
is  tho  basis  it  provides  for  promoting  the  attitude  among  students 
that  their  efforts  are  being  accurately  and  fairly  assessed. 

In  tho  outline  of  the  development  of  printed  tests  which  follows, 
it  will  first  bo  pointed  out  where  printed  tests  were  particularly  ap- 

__ 1  aii l:i  chapter  wn.i  written  by  Sgt.  Norman  OrnlT  with  tho  ni<ulstnnee  of  Cpl.  HaroM 
Ki-Ui’y  nn,l  Sgt.  Albert  Ilnatorf. 
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plicablc  and,  hence,  were  used  in  radar  oh>erver  training.  For  each 
of  the  four  classroom  subjects  of  tlie  course,  set  operation,  radar  navi¬ 
gation,  radar  bombing,  and  radar  intelligence,  printed  tests  served 
cither  or  both  of  two  functions:  (1)  Measuring  proficiency  in  solving 
problems  and  (2)  measuring  verbal  Knowledge  of  technical  informa¬ 
tion.  The  construction  of  printed  tests  will  be  discussed  with  refer¬ 
ence  to  sources  of  information  for  items,  preparation  of  items,  pro¬ 
visions  for  subjecting  items  to  expert  criticism,  weighting  the  several 
Auctions  of  a  test,  determining  time  limits,  scoring  formulas,  and 
format.  The  procedures  described  for  revisions  included  systematic¬ 
ally  compiling  and  reviewing  criticisms,  and  statistically  analyzing 
existing  forms.  The  extent  to  which  uniform  testing  was  achieved 
is  outlined  in  a  discussion  of  standardized  testing  conditions.  Cer¬ 
tain  difficulties  encountered  in  using  standardized  proficiency  tests 
arc  reported  and  interpreted  as  resulting  from  rapid  equipment  devel¬ 
opments  mul  curriculum  changes  in  the  newly-formed  nular  observer 
program.  The  remainder  of  tho  chapter  describes  eight  representa¬ 
tive  printed  proficiency  tests  administered  in  radar  observer  training. 
Each  of  these  is  described  in  terms  of  position  in  the  course,  subject 
matter  tested,  relation  to  earlier  achievement  tests,  specific  details  of 
content  and  construction,  and  data  from  statistical  analysis. 

USES  OF  STANDARDIZED  PROFICIENCY  TESTS  IN  RADAR 

OBSERVER  TRAINING 

Tlio  several  phases  of  radar  observer  training  presented  different 
problems  in  constructing  proficiency  measures.  It  was  realized  that 
the  skills  taught  in  some  parts  of  the  training  could  be  adequately 
measured  only  by  performance  checks.  Information  and  skills  taught 
in  other  sections,  on  tho  other  hand,  could  be  measured  by  printed 
tests.  In  general,  mnterial  taught  in  the  classroom,  which  comprised 
about  half  of  the  10-week  radar  observer  curriculum,  provided  tho 
basis  for  constructing  tests.  To  measure  acquisition  of  the  technical 
information  which  made  up  a  large  part  of  the  course,  tests  were  con¬ 
structed  consisting  wholly  of  verbal  questions.  Other  tests  w'ere  con¬ 
structed  with  problem-solving  items  to  measure  the  computational 
shills  taught  in  other  phases  of  the  training. 

Classroom  instruction  included  four  topics:  set  operation,  radar 
navigation,  radar  bombing,  and  radar  intelligence.  Classroom  in¬ 
struction  in  set  operation  included  the  location  and  function  of  tho 
controls  and  units  of  the  air-borne  radar  set.  Specific  operating  pro¬ 
cedures  included  starting  and  tuning  the  set,  calibrating  tho  range 
unit,  maintaining  maximum  definition  under  varying  conditions,  and 
locating  set  malfunctions.  Such  technical  information  provided  a 
readily-availnble  source  of  printed  test  items.  Consequently,  the  set 
operation  proficiency  tests  consist  of  information  items  testing  verbal 
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knowledge  of  functions  and  procedures.  It  was  recognized,  however, 
that  discrete  items,  concerned  with  procedure  present  the  student  v  ith 
:in  artificial  .situation.  Such  items  require  the  recall  of  an  element  of 
procedure  quite  outside  of  (he  usual  context  where  it  is  recalled  as 
part  of  an  actual  operating  sequence.  Also  artificial  are  the  verbal 
descriptions  of  symptoms  of  set  malfunction  used  in  questions  framed 

in  such  terms  as  ‘’What  should  yon  do  when _ ?” 

It  is  entirely  posiblc  that  a  student  could  respond  correctly  to  verbal 
descriptions  of  a  malfunction  and  yet  not  be  able  to  recognize  it  when 
actually  operating  the  set.  To  measure  set  operating  ability  under 
less  artificial  conditions,  the  bench  set  performance  checks  and  certain 
items  in  the  aerial  and  supersonic  checks,  described  in  Chapter  6,  were 
developed. 

The  radar  navigation  courses  consisted,  for  the  most  part,  of  the 
solution  of  navigation  problems  using  maps,  the  E  CB  computer, 
Weems  plotter,  and  dividers.  The  recognition  of  typical  radar  re¬ 
turns  was  also  taught,  as  well  as  the  application  to  the  task  of  naviga¬ 
tion  of  information  obtained  from  the  set.  The  radar  navigation  pro¬ 
ficiency  tests  included  both  problem-solving  and  information  items, 
primarily  the  former.  The  problems  presented  in  these  tests  included 
airplot  solutions  of  wind  problems,  determination  of  wind,  heading, 
and  track  on  the  vector  face  of  the.  E-GB  computer,  and  solution  of 
timc-ratc-distancc  and  altitude  problems  with  the  slide-rule  face  of 
the  E-GB  computer.  In  addition  to  solving  such  problems  as  were 
included  in  these  tests,  it  was  felt  desirable,  also,  to  measure  profi¬ 
ciency  in  navigating  typical  missions  which  required  the  student  to 
obtain  necessary  data  from  navigation  instruments  and  organize  tlm 
use  of  these  data.  Supersonic  trainer  and  aerial  performance  checks 
were  constructed  to  measure  performance  on  such  missions. 

In  radar-bombing  classes,  students  were  taught  theory  of  bombing, 
radar-bombing  procedures,  and  the  set  operation  procedures  involved 
in  radar  bombing.  Knowledge  of  those  subjects  was  measured  by 
multiple-choice  technical  information  items.  The  application  of  for¬ 
mulas  and  principles  to  determine  variables  used  in  bombing  was  also 
taught;  measurement,  here,  required  the  use  of  problem-solving 
items.  Such  items  dealt  with  computing  radius  of  turn,  determining 
necessary  drift  corrections,  and  predicting  absolute  altitude  over  the 
target. 

Again,  however,  the  procedure  on  an  actual  bombing  run  is  qnito 
different  from  solving  discrete  items  and  providing  answers  to  ver¬ 
bally  presented  problems.  The  radar  observer's  basic  source  of  in¬ 
formation  on  a  radar-controlled  bombing  run  is  a  complex  pattern  of 
returns  on  the  PPI  scope  which  moves  across  the  scope  and  becomes 
increasingly  detailed.  This  moving  complex  pattern  can  bo  repre¬ 
sented  only  very  roughly  by  a  series  of  scope  photographs  in  a  printed 
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tost.  Another  important  characteristic  of  actual  bombing  runs  is  that 
the  student  must  integrate  the  various  tasks  essential  to  the  bomb  run 
into  a  smoothly  functioning  system  which  permits  him  to  complete 
the  multitude  of  tasks  in  a  short  time.  Such  integration  of  tasks 
probably  cannot  be  tested  by  discrete  test  items.  To  measure  these 
aspects  of  bombing  proficiency  which  could  not  be  measured  by  printed 
tests,  the  bombing  sections  of  the  supersonic  trainer  and  aerial  per¬ 
formance  checks  were  developed. 

The  radar  intelligence  classes  consisted  of  the  presentation  of  fac¬ 
tual  information  regarding  radar  countermeasures,  priority  targets, 
survival  techniques,  and  target  studies.  This  phase  of  radar  training 
was  probably  measured  adequately  by  the  single  type  of  proficiency 
measure  used:  the  printed  test  comprised  of  technical  information 
items. 

On  the  basis  of  the  foregoing  considerations,  a  battery  of  printed 
tests  and  performance  checks  was  prepared  for  the  radar  observer 
proficiency  measurement  program.  This  program  was  of  interest  in 
that  it  represented  the  first  timo  in  aircrew  training  tlint  a  complete 
battery  of  proficiency  measures  presented  by  a  psychological  research 
organization  was  approved  and  adopted  for  use  in  all  the  training 
stations  of  an  aircrew  specialty.  The  six  performance  checks  included 
in  this  battery  are  described  in  chapter  6  of  this  report.  Five  printed 
tests  completed  the  battery.  These  included  a  test  given  at  an  inter¬ 
mediate  stage  of  training  for  each  of  the  four  subjects  in  the  course 
and  a  final  comprehensive  examination  sampling  all  four  of  the  sub¬ 
jects.  The  intermediate  tests  were  titled  ns  follows:  Set  Operation 
Intermediate  Test,  Radar  Navigation  Intermediate  Test,  Radar 
Bombing  Intermediate  Test,  and  Radar  Intelligence  Intermediate 
Test.  The  comprehensive  examination  consisted  of  two  test  booklets 
called  Final  Test  I  and  Final  Test  II,  which  were  administered  dur¬ 
ing  a  single  testing  period. 

GENERAL  PROCEDURES  USED  IN  DEVELOPING  PRINTED 

TESTS 

Preliminary  Drafts 

The  first  step  in  constructing  printed  proficiency  tests  consisted  of 
collecting  the  subject  matter  from  ail  possible  sources  for  each  of  the 
four  subjects  of  the  radar  observer  course :  set  operation,  radar  naviga¬ 
tion,  radar  bombing  and  radar  intelligence.  Course  outlines,  indi¬ 
vidual  lecture  outlines,  and  lecture  notes  of  personnel  from  the  Radar 
Project  enrolled  in  the  course*  were  assembled.  Information  from 


*  Radar  project  pernonnel  who  completed  requirements  of  the  rndnr  training  curriculum 
wore:  l.t.  William  A.  McClelland.  I.t.  Georgo  S.  Klein,  Tech.  Sj;t.  Sanford  J.  Mock,  St.  Sgt 
HI cluird  T.  Mitchell,  Sgt.  Nathaniel  L.  Gage.  Sergeant  QralT,  Cpl.  Nelson  R.  Nall,  Cpl.  WU- 
bert  II.  Sehwotzor,  and  l’fc.  Dwanc  It.  Collins. 
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tlu-.-e  soilin' was  assigned  to  one  of  the  four  categories  of  subject  mat- 
ler  and  items  were  lenta1  ivcly  drawn  up  for  elements  of  information  in 
each  category.  Existing  tests  were  examined  and  some  items  were 
revi.-ed  and  retained. 

d  im  question  of  the  relative  weight  to  lie  assigned  to  each  of  the  four 
subjects  in  the  course  was  decided  by  training  authorities.  The  as¬ 
signed  weights  were  followed  in  constructing  the  te.-ts  by  apportioning 
more  items  to  one  subject  than  another.  A  difficulty  in  the  use  of  this 
method,  arising  primarily  in  the  construction  of  technical  knowledge 
sections,  was  the  lack  of  sufficient  Lest  item  material  for  the  more 
heavily  weighted  topics.  This  was  especially  true  of  two  subjects, 
radar  bombing  and  radar  navigation,  which  required  many  instruc¬ 
tional  hours  in  the  course  for  the  presentation  of  testable  subject  mat¬ 
ter.  Less  difficulty  was  encountered  in  expanding  problem-solving 
sections  to  increase  their  weight.  Emphasis  was  placed  on  the  impor¬ 
tant  navigation  and  bombing  techniques  by  presenting  each  problem 
a  number  of  times  but  with  altered  data  from  item  to  item. 


Review  and  Criticism 


After  preliminary  drafting,  all  items  were  submitted  to  radar  experts 
for  review  and  criticism.  First,  the  individual  items  were  reviewed 
by  instructors  who  had  prepared  the  lectures  from  which  item  ma¬ 
terial  was  taken.  Revisions  were  made  on  the  basis  of  resulting  crit¬ 
icism  and  a  first  draft  of  the  proposed  test  was  constructed.  This  first 
draft  was  tb.cn  presented  to  an  assembly  of  all  instructors  Leaching  the 
pertinent  subject.  It  was  later  found  to  he  more  efficient  to  submit 
proposed  tests  to  school  authorities  rather  than  to  instructors.  This 
had  the  advantage  of  obtaining  criticism  from  individuals  not  involved 
in  teaching  the  source  material  and  the  administrative  criticism  in¬ 
sured  an  acceptable  relation  of  test  content  to  graduation  standaida. 

The  methods  just  described  for  obtaining  criticism  were  sufficient 
for  technical  information  items.  For  problem-solving  items,  it  was 
necessary  to  obtain  further  judgments  from  expert  radar  observers 
regarding  correct  answers  and  desirable  tolerances  for  incorrect  alter¬ 
natives.  To  facilitate  obtaining  these  decisions,  a  sy-  tem  was  devoid]  ed 
which  made  use  of  three  panels  of  five  experts  each.  The  members  of 
each  panel  individually  solved  the  test  items  after  which  the  live  ex¬ 


perts  compared  results  and  agreed  upon  answers  and  mislead  toler¬ 
ances.  When  the  decisions  of  the  three  panels  were  compared,  they 
u •  ually  rellected  sufficient  agreement  to  confirm  the  adequacy  of  the 
svpa  l  ately  determined  values.  As  a  final  check  to  insure  accnrm  y,  the 
Ruhr  Project  test  const  ruction  teams  carefully  reworked  each  prob¬ 
lem.  An  incidental  benefit  derived  from  the  panel  system  was  that  tho 
working  time  required  by  the  experts  in  the  original  solution  of  tho 
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items  provided  an  approximate  time  limit  for  the  first  administration 
of  the  test. 

Test  Format 

Because  the  alternative  choices  in  both  the  problem-solving  and 
technical  information  items  were  brief  and  required  little  space,  items 
were  arranged  on  the  pages  of  the  test  booklets  in  two  columns.  '1  his 
accomplished  a  saving  in  space  and  gave,  a  desirable  compactness  to 
each  item.  Answers  were  marked  on  a  separate  answer  sheet  which 
allowed  for  machine  scoring.  In  assembling  the  various  sections  of 
each  to.-l,  Limed  problem-solving  sections — the  only  sections  requiring 
the  use  of  equipment — were  placed  first  in  the  test  booklets.  This  ar¬ 
rangement  facilitated  proetoring  by  making  it  difficult  for  students  to 
work  unnoticed  on  problem-solving  sections  after  time  limits  had 
elapsed.  It  also  facilitated  administration  since  few  students  finished 
the  speeded  sections  before  the  time  limits  were  up. 

Time  Limits 

It  has  been  mentioned  that  time  limits  for  initial  administrations  of 
problem-solving  sections  were  determined  on  the  basis  of  the  time  re¬ 
quired  by  panels  of  experts  to  solve  the  problems.  These  time  limits 
were  used  on  the  assumption  that  they  would  allow  very  few  students 
to  complete  the  problem-solving  sections.  In  assigning  time  limits  for 
technical  information  sections,  on  the  other  hand,  the  attempt  was  made 
to  allow  all' students  to  try  every  item.  The  practice  of  speeding  the 
problem-solving  sections  met  with  resistance  from  some  school  au¬ 
thorities,  who  thought  that  tasks  measured  in  these  sections  were  not 
solved  under  time  pressure  in  actual  radar  bombing  and  navigation. 
Whereas  some  authorities  felt  that  accuracy,  not  speed,  was  most  im¬ 
portant  in  solving  bombing  and  navigation  problems,  others  believed 
that  speedy,  systematic  handling  of  the  problems  was  paramount. 
Among  members  of  the  Radar  Project  it  was  felt,  on  the  basis  of  ob¬ 
servations  of  trained  radar  observers  at  work,  that  both  speed  and  ac¬ 
curacy  should  be  measured.  Another  justification  for  speeding  some 
sections  was  that  a  section  scored  only  in  terms  of  accuracy  would 
provide  loss  differentiation  among  students  than  would  the  same  sec¬ 
tion  given  with  an  appropriate  time  limit.  The  latter  would  discrim¬ 
inate  between  those  students  who  require  a  great  deal  of  time  to  get 
accurate  answers  and  those  who  work  both  quickly  and  accurately. 

Scoring  Formula 

AH  tests  were  scored  simply  in  terms  of  the  number  of  correct 
responses.  The  raw  scores  were  converted  to  ccntile  scores  by  means 
of  conversion  tables  constructed  from  tho  data  accumulated  from  a 
number  of  classes.  No  attempt  was  made  in  the  scoring  formula  to 
correct  for  guessing.  Tho  use  of  scoring  formulas  for  this  purpose 
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would  have  been  complicated  becmiM*  not  nil  items  lintl  the  same  num¬ 
ber  of  alternative  responses,  the  number  ranging  from  two  to  five. 

liases  for  Revisions 

The  urgency  of  installing  the  proficiency  testing  program  made  it 
impractical  to  administer  any  of  the  tests  experimentally  prior  to 
(heir  publication.  Consequently,  first  forms  of  the  printed  tests  were 
put  into  use  without  being  subjected  to  the  customary  statistical 
analysis  to  detect  item  defects.  Revisions  were  based  upon  statistical 
analysis  of  existing  forms,  systematically  gathered  subjective  criti¬ 
cisms,  and  changes  in  the  curriculum. 

Inspection  of  the  means  and  standard  deviations  of  the  various 
sections  of  a  test  and  of  the  test  as  a  whole  often  indicated  necessary 
revisions.  For  example,  these  statistics  for  a  speeded  problem-solving 
section  of  a  test,  part  A  of  Final  Test  I,  indicated  that  this  section  did 
not  contribute  to  the  total  test  score  in  proportion  to  the  time  spent 
in  its  administration.  To  increase  the  number  of  items  in  the  section 
wihout  adding  to  its  testing  time,  existing  items  which  required  inter¬ 
mediate  steps  for  their  solution  were  broken  down  into  separate  items, 
each  requiring  an  answer.  Also,  answers  which  were  given  in  terms 
of  two  independent  quantities  were  divided  between  two  items.  For 
example,  instead  of  asking  in  a  single  item  for  a  wind  solution  in  terms 
of  wind  direction  and  wind  force,  the  student  was  asked  in  one  item 
for  wfind  direction  and  in  another  for  wind  force.  The  results  of 
using  the  two  methods  on  part  A  of  Final  Test  I  are  described  on 
pagG  81.®  Tho  technique  of  making  separate  items  out  of  the  inter¬ 
mediate  steps  in  the  solution  of  a  complex  problem  was  first  employed 
by  Psychological  Research  Project  (Navigator)  where  13  problem¬ 
solving  items  were  expanded  to  119  items. 

The  distribution  of  raw  scores  for  speeded  sections  also  served  as 
a  guide  for  determining  the  adequacy  of  time  limits.  The  distribu¬ 
tions  from  early  administrations  of  most  of  the  speeded  sections  were 
negatively  skewed,  most  of  the  scores  being  concentrated  at  the  upper 
end  of  the  scale.  This  lack  of  discrimination  among  students  at 
the  upper  levels  of  proficiency  was  taken  to  mean  that  the  time  limits 
"'ere  too  long.  To  increase  differentiation  among  the  higher  scoring 
students,  the  time  limits  were  shortened  and  tho  distributions  became 
more  symmetrical. 

Item  analyses  in  terms  of  difficulty  level  and  internal  consistency 
were  employed  to  yield  data  for  test  revisions.  Item  difficulty  was 
measured  by  the  percentage  of  students  attempting  an  item  who  also 
got  it.  right.  The  correlation  of  eacli  item  with  the  total  test  score 
was  found  by  computing  a  phi  coefficient.  This  coefficient  was  based 


i 


*In  table  B.8  nro  presented  the  means  nnd  stnndard  deviations  of  pnrt  A  of  l’lnnl  Test 
before  and  ntter  splitting  two-answer  Items  Into  two  separate  Items. 
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upon  (he  relative  percentages  getting  the  item  right  in  the  upper  and 
lower  50  percent  of  the  group  divided  on  the  basis  of  total  test  score. 
Items  which  were  shown  to  be  too  difficult  or  too  easy  were  eliminated 
or  revised,  as  were  items  having  zero  or  negative  correlation  with  the 
total  test  score.  In  some  instances,  however,  such  items  were  re¬ 
tained  because  of  the  importance  of  their  content  to  an  .adequate 
sampling  of  the  subject  matter. 

An  attempt  was  made  to  compile  systematically  the  criticism  made 
by  instructors  and  students  during  the  period  of  administering  a  given 
form.  These  criticisms,  usually  dealing  with  item  clarity  or  the  ade¬ 
quacy  with  which  a  test  sampled  the  subject  matter,  were  considered 
and  items  wore  adjusted  where  necessary.  Changes  and  developments 
in  airborne  radar  equipment  resulted  in  changes  in  the  subject  matter 
taught  in  the  course,  primarily  in  set  operation.  These  changes  fre¬ 
quently  made  it  necessary  to  delete  obsolete  material  and  incorporate 
new  material  in  the  test3. 

STANDARDIZATION  OF  TEST  ADMINISTRATION 

Several  steps  were  taken  to  promote  the  standardized  test  adminis¬ 
tration  which  is  essential  to  the  success  of  a  proficiency  evaluation 
program.  The  examiners  who  administered  the  tests  were  selected 
for  the  task  by  the  school  authorities.  They  were  given  indoctrina¬ 
tion  and  instruction  in  methods  of  test  administration  by  personnel  of 
the  Radar  Project.  Detailed  standardized  directions  for  administra¬ 
tion  were  provided  for  each  test.  These  directions  prescribed  ap¬ 
proved  procedures  for  distributing  and  collecting  test  materials  and 
timing  the  various  sections  of  the  test,  and  included  directions  to  the 
students  which  the  examiner  read  verbatim.  All  testing  was  carried 
out  in  specially  designated  rooms  at  each  training  station.  Adequate 
working  space  was  provided  each  student  and  necessary  precautions 
were  taken  to  insure  independent  work  during  the  test.  All  tests 
were  scored  under  the  immediate  supervision  of  the  Radar  Project, 
and  conversion  tables  and  rosters  of  raw  and  converted  scores  were 
prepared  for  the  training  authorities. 

As  in  all  testing  situations,  motivation  played  an  important  part 
in  the  test  results.  Radar  observer  students  had  already  received 
their  commissions  and  aircrew  specialty  ratings  either  as  bombardiers 
or  navigators  and  it  appeared  that  many  of  them  did  not  care  whether 
or  not  they  gi  duated  from  radar  observer  training.  Illustrative  of 
the  motivation  problem  is  the  following  incident  which  occurred 
after  the  cessation  of  hostilities  in  the  Pacific  on  14  August  1945.4 
On  23  August,  class  45-34  at  Langley  Field,  consi  ;ing  of  39  students, 
took  the  final  examination  and  13  students  failed.  The  following 
week,  the  succeeding  class,  45-35,  which  included  six  of  the  previous 

4  S t a t lm leal  analysis  of  tills  problem  was  conducted  by  Cpl.  Hyman  Sofer. 


64 


failures,  was  promised  a  week-end  leave  for  “passing”  the  examina¬ 
tion.  With  this  goal,  only  2  men  in  45  failed.  The  data  for  the  two 
classes  are  presented  in  table  5.1.  In  addition,  data  for  class  45-33 


Table  5.1. — Effect  of  motivation  on  raw  scores  from  speeded  and  nonspeeded 

tests  ‘ 


Class 

N 

Final  tost  I, 
Pln-B 
(speeded) 

Final  test  11, 
ril)-n  (nnn- 
si>eedi'd) 

Total  of  final 
tests  1  and  11 

Mean 

SD 

Mean 

SD 

Mean 

8D 

<5-33  (pro-VJ-day) . - . 

<5-31  (post-VJ-d.iy,  unmotivated) . 

<5-35  (post-VJ-day,  motivated) . 

31 

30 

<5 

62.  55 
51.31 
61.33 

10. 02 
10.65 
11.  3< 

jj^^j 

mm 

■ 

i  Bused  upon  administrations  at  Langley  Field. 


are  presented  to  show  the  level  of  performance  for  students  who  com¬ 
pleted  their  training  before  the  Japanese  surrender.  It  will  be  noted 
that  the  specially  motivated  class  attained  the  same  level  as  the  pro¬ 
surrender  group  and  that  the  low  score  of  the  poorly  motivated  group 
was  duo  primarily  to  poor  performance  on  the  speeded  section  of  the 
test. 


DIFFICULTIES  EXPERIENCED  IN  THE  USE  OF  STANDARD¬ 
IZED  PROFICIENCY  TESTS 

The  usual  difficulties  encountered  in  installing  and  administering 
a  battery  of  standardized  proficiency  tests  were  intensified  by  the 
urgency  and  acceleration  of  the  radar  training  program.  At  the 
time  the  program  was  initiated,  progress  in  the  development  of  air¬ 
borne  radar  equipment  was  rapid.  Equipment  improvements  which 
required  new  operating  techniques  were  being  constantly  introduced. 
These  changes  inevitably  produced  difficulties  in  applying  a  given 
standardized  test  to  students  over  any  considerable  period  of  time.  In 
several  instances,  to  eliminate  obsolete  subject  matter,  it  was  necessary 
to  delete  items  without  replacing  them.  This  was  particularly  true  of 
the  radar  intelligence  course. 

Because  of  the  changes  in  equipment,  students  were  often  trained  on 
sets  that  differed  from  those  for  which  proficiency  tests  had  been  con¬ 
structed.  Further,  the  training  literature  often  lagged  behind  the 
appearance  of  the  new  equipment.  The  development  of  lectures  ex¬ 
plaining  new  equipment  and  the  attainment  of  proficiency  by  instruc¬ 
tors  in  new  procedures  was  consequently  slow.  These  things  combined 
to  yield  inconsistencies  between  the  subject  matter  taught  and  the  con¬ 
tent  of  the  printed  tests. 

On  several  occasions,  there  was  disagreement  among  the  schools  as 
to  correct  operating  procedures  and  the  emphasis  t.iat  should  be  placed 
on  various  phases  of  the  course.  For  example,  or.o  school  trained  its 
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students  in  navigation  w it li  the  view  of  graduating  students  who  were 
prepared  to  take  over  the  navigation  function  in  emergency  combat 
situations.  Another  school  emphasized  set  operation  and  bombing 
and  required,  as  the  only  navigation  function,  the  plotting  of  fix  in¬ 
formation  for  the  dead-reckoning  navigator.  At  several  schools  stu¬ 
dents  were  trained  to  obtain  winds  by  the  target-timing  method  when 
this  method  did  not  appear  in  the  curriculum  of  the  other  schools. 

Unfortunately,  training  authorities  tended  to  desire  test  items  which 
could  be  passed  by  most  of  the  students.  This  feeling  arose  from  a 
conception  of  the  minimum  information  required  of  a  radar  observer 
in  combat  and,  apparently,  also  from  a  desire  to  nial  e  the  results  from 
a  particular  school  seem  to  indicate  a  high  level  of  proficiency.  How¬ 
ever,  it  was  possible  to  include  difficult  items  in  the  tests  by  explaining 
their  function  in  discriminating  among  students  at  the  upper  levels 
of  proficiency. 


DESCRIPTION  OF  TYPICAL  PRINTED  TESTS  USED  IN  THE 
RADAR  TRAINING  PROGRAM 


Eight  proficiency  tests  which  were  used  in  the  radar  training  pro¬ 
gram  are  described  below.  Each  test  will  be  discussed  in  terms  of 
its  function  in  the  training  program,  its  development,  item  content, 
details  of  administration,  important  revisions,  and  statistical  findings. 
First  will  be  presented  the  four  tests  given  at  intermediate  points  in 
the  training,  measuring  proficiency  in  each  of  the  four  subjects  of  the 
course.  These  intermediate  tests,  in  order  of  administration  in  the 
course  and  presentation  below,  are:  Set  Operation  Intermediate  Test, 
Radar  Navigation  Intermediate  Test,  Iludar  Bombing  Intermediate 
Test,  and  Radar  Intelligence  Intermediate  Test.  Next  nro  presented 
the  two  parts  of  tho  comprehensive  examination,  Final  Test  I  and 
Final  Test  II,  administered  at  the  end  of  the  course.  These  are  fol¬ 
lowed  by  a  description  of  a  comprehensive  examination,  final  test  for 
All/APQ-7  set,  prepared  to  measure  proficiency  on  a  recent  airborne 
radar  set.  Finally  described  is  the  Navigation  Proficiency  Test, 
not  a  radar  test,  which  was  used  as  part  of  the  selection  battery  given 
to  candidates  for  radar  observer  training. 


Set  Operation  Intermediate  Test 

Tho  set  operation  intermediate  test 3  consisted  primarily  of  technical 
information  items  measuring  knowledge  of  the  operation  of  radar 
equipment,  ski’’  i  analyzing  equipment  malfunctions,  and  knowledge 
of  auxiliary  equipment  used  in  radar  navigation  and  bombing.  Tho 
various  forms  of  the  test  contain  approximately  100  multiple  choice 
items  and  requiro  about  an  hour  for  administration.  A  set  operation 


1  MrHt  forms  of  these  tests  were  developed  by  Sgt.  Grnff,  nnd  Sgt.  Krlcilt.  Revised 
forma  were  prepared  by  S/Sgt.  Mitchell  uiul  Sgt.  Gerald  S.  Ilium. 
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intermediate  test  was  admini-tered  to  nulur  observer  students  at  ap¬ 
proximately  the  middle  of  the  ten-week  course  after  the  classroom 
study  of  set  operation  had  been  completed. 

Most  of  the  item  material  for  the  first  form  of  this  test  was  obtained 
from  the  radar  operator  achievement  examination  for  AN/APS-15, 
prepared  at  the.  request  of  the  Psychological  Section,  Medical  Re¬ 
search  Division,  Air  Surgeon’s  Office,  by  a  project  of  the  National  De¬ 
fense.  Research  Committee.  This  test  was  given  by  the  A  irerew  Evalu¬ 
ation  and  Research  Detachment  No.  1  in  the  selection  of  pathfinder 
crews  for  the  Eighth  Air  Force.  Revisions  were  made  by  the  De¬ 
tachment  and  the  resulting  form,  after  supplementation  by  the  radar 
project,  became  form  A  of  Set  Operation  Intermediate  Test,  Plk-A 
and  PlLr-A. 


Various  forms  of  the  test  developed  by  the  radar  project  contained 
from  7G  to  120  items.  None  of  the  forms  was  speeded.  Each  re¬ 
quired  50  to  75  minutes  testing  time.  No  variations  from  the  general 
procedure  of  developing  the  test  items  were  employed.  Although, 
most,  of  the  items  are  of  the  verbal  knowledge  type,  some  may  be  classi¬ 
fied  ns  problem-solving  items,  since  they  require  the  application  of 
theory  to  the  solution  of  problems. 

Samples: 


17.  If  there  is  no  sweep  or  spot  on  the  scope,  the  trouble  is  a  blown 
fuse  in  the 

17-A  Synchronizer. 

17-B  Range  unit. 

17-C  High  voltage  rectifier  circuit. 

17-D  Low  voltage  rectifier  circuit. 

17- E  Main  control  box. 

18.  Given: 

Range  unit  “on.”  Altitude  2.2  miles. 

Sweep  delay  at  30  miles.  Computer  drum  reads  4.9. 

Bombing  circle  on  target 
Find  :  What  is  the  slant  range  to  the  target! 

13-A  30.9. 

18- B  32.7. 

18-C  32.9. 

18-D  34.9 
18-E  37.1. 


In  the  set  operation  intermediate  tests  developed  for  the  Eastern 
Tra  ining  Command  schools,  the  booklets  were  divided  into  sections 
on  the  basis  of  the  following  areas  of  subject  matter:  normal  oper¬ 
ating  procedures,  locating  malfunctions,  and  use  of  auxiliary  equip¬ 
ment.  For  Western  Training  Command  schools,  no  division  into 
parts  was  made. 
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Revisions  of  set  operation  tests  were  necessitated  primarily  by  tho 
introduction  of  new  sets  and  equipment  into  the  training  program. 
"When  theso  revisions  were  made,  statistical  analyses  of  previous  forms 
were  employed.  In  all,  six  set  operation  intermediate  tests  were  de¬ 
veloped.  The  first  form  produced  by  the  Radar  Project  consisted 
of  two  sections  combined  into  a  single  booklet,  Pllc-A  and  P1L-A  for 
tlio  AN/APS-15  and  AN/APS-15A,  respectively.  This  test  was  re¬ 
placed  by  Form  13  of  PI  1—13  when  instruction  on  the  AX-APS-15  was 
eliminated  from  the  course.  Two  forms  were  constructed  for  the 
AN/APQ-13,  Plm-A  and  V  Plm-A.  When  the  AN/APQ-7  pro¬ 
gram  needed  a  set  operation  test,  the  Radar  Project  was  so  absorbed 
with  meeting  tho  demands  for  test  revisions  from  other  schools  and 
with  supervision  of  performance  check  administration  that  aid  from 
tho  NDRC  Project  was  requested.  The  temporary  test  produced,  at 
that  time,  Radar  Operator’s  Proficiency  Examination  for  AN/APQ-7 
was  later  replaced  by  radar  project  revisions,  W  Plt-A  and  W  Plt-B, 
which  incorporated  new  test  items  from  the  materials  in  the  growing 
curriculum. 

With  ono  exception,  the  distributions  of  raw  scores  from  the  various 
forms  were  negatively  skewed.  Tho  distribution  statistics  given  in 
table  5.2  indicate  that  tho  tests  were  fairly  easy. 


Taih.k  5.2. — (uni  standard  deviations  of  distributions  of  raw  scores  from 
various  forms  of  the  set  operations  intermediate  test 1 


School 

Classes 

Form 

Num¬ 

ber 

Items 

Tlrao 

N 

Mean 

SD 

1 . 

All  cases . 

V  Plm-A . 

120 

Minutes 

SO 

12.07 

2 . 

All  tiws . 

Plm-A . 

100 

OO 

10.  32 

3  . 

■16*18  through  15-‘J0 . 

I’l  k-A  auU  I’ll-A . 

101 

7.5 

j^eTkT 

mritMthT 

10.  00 

3  . 

All  CU50S . 

l’ll-U . . 

80 

CO  I 

700 

70.  80 

10.07 

■  School  1  Is  Victorville,  2  Is  Iloca  Roton,  and  3  Is  Langley  Field. 


Tho  odd-even  reliability  for  form  Plk-A  and  Pll-A  is  0.G0  which 
corrects  by  the  Spearman-Brown  formula  to  0.74.  This  form  has  104 
items  and  a  timo  limit  of  75  minutes.  The  mean  and  standard  devia¬ 
tion  of  the  odd  scores  uro  38.11  and  4.9G  respectively.  The  mean  and 
standard  deviation  of  the  even  scores  arc  35.45  and  5.03  respectively. 

Item  analysis  of  difficulty  level  based  on  431  cases  for  form  Plm-A, 
which  yielded  an  approximately  normal  distribution  of  raw  scores, 
shows  udequato  item  difficulty  lovcl.  The  median  item  difficulty  for 
the  upper  half  of  the  group  was  G8  percent  passing  and  for  the  lower 
half  was  40  percent.  This  form,  which  has  100  items  with  a  time  limit 
of  00  minutes,  yielded  a  median  phi  of  0.10  when  analyzed  for  internal 
consistency  using  the  same  434  cases.  Computation  of  phi  was  based 
upon  the  upper  and  lower  50  percent  of  the  group. 
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Kiulnr  Navigation  Intermediate  Test 

The  radar  navigation  intermediate  test8  was  designed  to  measure 
proficiency  in  solving  typical  radar  navigation  problems,  and  verbal 
knowledge  of  the  techniques  of  radar  navigation.  Tim  various  forms 
of  the  test  contain  from  (15  to  90  multiple-choice  items  and  require  from 
65  to  90  minutes  for  administration.  A  navigation  intermediate  test 
was  administered  upon  completion  of  classroom  training  in  radar  navi¬ 
gation,  which  was  given  during  the  first  three-quarters  of  the  course. 

The  materials  for  the  items  of  the  early  forms  were  obtained  by  the 
radar  project  from  existing  instructors’  quizzes,  navigation  manuals 
and  lectures,  and  suggestions  of  aerial  and  ground  school  instructors. 
Helpful  ideas  for  adapting  navigation  problems  to  multiple-choice 
items  were  gained  from  the  proficiency  tests  developed  by  the  navigator 
project. 

Most  of  the  items  in  the  radar  navigation  intermediate  tests  were 
of  tho  problem-solving  type.  Since  navigation  problems  require  the 
uso  of  a  rather  complex  set  of  data,  it  was  arranged,  in  order  to  save 
testing  time  and  space  in  tho  test  booklet,  to  have  two  or  more  items 
based  upon  a  single  set  of  basic  data.  To  simulate  inaccuracies  which 
commonly  enter  into  navigation  instruments  and  data,  erroneous  ma¬ 
terials  were  occasionally  introduced  into  the  basic  data.  Also,  it  was 
felt  that  by  incorporating  erroneous  information,  the  student  would 
bo  required  to  use  judgment  comparable  to  the  judgment  he  would 
employ  in  the  air  when  rejecting  a  poorly  plotted  fix.  These  techniques 
are  illustrated  in  the  following  sample  item.  It  will  be  noted  that  item 
28  is  dependent  upon  items  26  and  27  insofar  as  the  student  must  use 
the  selected  alternatives  to  tho  latter  items  to  solve  item  28. 

Sample : 

Some  of  the  fixes  are  incorrect,  consequently  all  six  fixes  should  be 
plotted  for  each  problem.  Tho  times  for  the  first  and  last  fixes  are 
always  accurate.  Questions  2G-28  are  parts  of  the  same  problems. 

Given:  Time,  bearing,  and  horizontal  range  for  six  wind  run  fixes. 

102804,  199°,  14  n.  m. 

102950,  193°,  11  n.  m. 

103430,  071°,  8  n.  m. 

103515,  062°,  11  n.  m. 

26.  Find :  True  course. 

26-A  038°. 

2G-B  042°. 

2G-C  208°. 

2G-D  218°. 

2G-E  222°. 


*  Developed  by'Sgt.  Graff.  S«t.  Kriedt,  and  Cpl.  Koch. 


103603,  056°,  14  n.  m. 
True  heading,  212°. 
True  aif  peed,  152  K. 
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27.  Find  :  Ground  speed. 

2 7- A  170  IC. 

27-13  ISO  K. 

27  C  1 00  IC. 

27- U  200  IC. 

27- 10  210  IC. 

28.  Find  :  'Wind  (use  (.lie  GS  and  TC  selected  as  correct  for  question 
26  and  27). 

28  -A  010750  IC. 

28- 13  O217M  IC. 

28-C  05F/50  IC. 

28-D  0517*14  IC. 

28-E  056750  K. 


Early  forms  included  items  on  scope  interpretation,  radar  unit  func¬ 
tions,  ’  idar  beacon  navigation,  and  E-6I3  computer  problems.  Later 
forms  were  more  definitely  divided  into  separate  sections.  The  most 
heavily  weighted  section  was  the  theory  and  technique  of  radar  nav¬ 
igation,  course  determination,  and  drift  determination.  In  the  vari¬ 
ous  forms,  this  first  section  included  from  28  to  72  items  given  in  30-  to 
70-minute  airspeeded  sessions.  Next  was  presented  a  speeded  section 
of  from  15  to  18  items  requiring  10  to  18  minutes  and  measuring  abil¬ 
ity  to  use  the  E-6B  computer  in  determining  wind  direction  and  force, 
true  heading,  true  course,  ground  speed,  ground  range,  and  ETA.  Ten 
airplot  problems  similar  to  those  described  on  page  80/for  Final  Test  I 
were  presented  in  another  speeded  section,  with  a  time  limit  of  20 
minutes.  For  the  airplot  section,  this  test  incorporated  a  standard 
expendable  Mercator  chart  the  size  of  the  test  booklet  upon  which  the 
student  plotted  his  own  latitude  and  longitude  data.  An  additional 
12-item  section,7  employed  only  in  the  Western  Flying  Training  Com¬ 
mand  schools,  was  used  for  measuring  ability  in  target  timing  wind 
determination.  This  speeded  section  had  a  time  limit  of  12  minutes. 

Original  forms  of  the  radar  navigation  intermediate  test  emphasized 
the  airplot  method  of  determining  winds  as  was  done  in  the  schools 
of  the  Eastern  Technical  Training  Command.  Important  revisions 
were  necessitated  by  the  introduction  in  Western  Flying  Training 
Command  schools  of  the  target  timing  method  of  determining  winds. 


Three  forms  were  constructed  for  the  eastern  schools :  Plh-A  and  Plh- 
13  for  the  AN/APS-15  set  and  Pli-A  for  the  AN/APS-15A  set. 
Throe  more  forms  were  constructed  for  the  AN/APQ-13  used  at  the 
western  schools:  Plj-A  and  V  Plj-A  for  use  at  Victorville,  and 
W  Plj-A  for  use  at  Williams  Field. 


*  Preliminary  work  on  this  section  was  conducted  nt  Boca  Raton,  Fin.,  by  Sgt.  Graff 
nnd  Sgt  Krlcilt.  The  latter  completed  t lie  section  In  the  Western  Flying  Training  Com¬ 
mand  station  at  Williams  Field- 
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The  moans  and  slamlard  deviations  for  t lie  part  and  the  total  scores 
of  form  Plli-B  are  presented  in  table  5.3.  The  extent  to  which  each 
section  contributed  to  the  total  score  may  be  estimated  from  the  stand¬ 
ard  deviations. 


T.vm.K  — Means  anil  shtnilanl  davit. lion#  of  distributions  of  part  and  total 
raw  scores  from  form  Pth-H  of  the  radar  navigation  intermediate  tcstr 
administered  at  Langley  Field 


Chutes 

l’nrt 

Numb*  r 
itvins 

Timo 

>J 

Mean 

SI) 

I . 

45 

Minutfs 

35 

709 

31.  33 

7.  37 

II . . . 

15 

10 

075 

8.  70 

2.  85 

Ill . 

10 

20 

05 

075 

4.  03 

1.92 

Total . 

70  1 

075 

4 18 

0.20 

Total . 

70 

05 

959 

43.  24 

8.84 

Two  reliability  studies  were  made  of  form  Plli-B.  The  first  con¬ 
sisted  of  an  odd-even  reliability  computed  for  part  A,  an  unspeeded 
section  containing  45  items  with  a  time  limit  of  35  minutes.  The 
odd-even  coefficient  based  on  179  cases,  Langley  Field  classes  45-19 
through  45-33,  is  0.61  which  corrects  by  the  Spearman-Brown  formula 
to  0.76.  The  mean  and  standard  deviation  of  the  odd  scores  are 
14.66  and  3.18,  respectively.  The  mean  and  standard  deviation  of  the 
even  scores  are  14.48  and  3.12,  respectively.  Odd-even  reliability 
coefficients  were  not  computed  for  the  other  two  parts  of  this  test 
because  they  were  speeded. 

The  second  reliability  study  consisted  of  determining  a  test-retest 
reliability  for  the  parts  and  total  score  of  form  Plli-B.  The  results, 
presented  in  table  5.4  are  based  upon  only  one  class  of  27  students. 
The  reliability  coefficient  for  the  total  test  was  found  to  be  0.64. 


Table  5.4.— Tcst-rctcst  reliability  coefficients  for  part  and  total  raw  scores  of 
radar  navigation  intermediate  test,  Fllv-B1 


Part 

,  N 

Mean- 

test 

Mean- 

retest 

SD-tcst 

SD-retcst 

r« 

SE.-.Off 

I  . 

27 

27.09 

32  74 

0.97 

3.31 

0.  59 

<120 

II . 

27 

8.  90 

11.52 

3.  40 

2.23 

.33 

.20 

HI . 

27 

3.  33 

5.  55 

1.05 

2  10 

.39 

.20 

Total . 

27 

39.  M) 

49. 18 

9. 06 

4.55 

.01 

.20 

1  n.v.'il  iip'.n  l.nnrli-y  Field  classes  -li-17;  ai  percent  were  bombardiers  and  the  remainder  were  navigators. 
Test  mid  retest  wi  re  :  epamted  by  nn  interval  of  1  week. 


Item  analysis  yielded  tho  difficulty  level  and  internal  consistency 
statistics  presented  in  table  5.5.  For  each  part  of  form  Plj-A  the 
median  difficulty  level  and  the  median  phi  coefficient  are  given  for 
tho  upper  and  lower  halves  of  the  group  divided  on  the  basis  of  total 
test  score.  Statistics  concerning  difficulty  level,  which  ranges  from  49 
to  87  percent  passing,  indicate  that  the  items  in  both  the  technical- 
information  section  and  problem-solving  sections  were  answered  cor- 
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redly  by  most  of  the  stiub'nts.  Of  considerable  interest  in  table  5.5 
is  the  discrepancy  in  didicully  level  between  school  1  and  school  2. 
The  students  from  school  1  were  mostly  navigators,  while  the.  students 
from  school  2  were  mostly  bombardiers.  Despite  the  fact  that  most  of 
the  developmental  work  on  this  form  of  the  test  was  conducted  at 
school  2,  its  students  consistently  scored  lower  than  students  from 
school  1. 


Taih.k  5.5. — Median  difficulty  level  for  upper  and  loircr  scoring  groups  and 
median  phis  for  the  parts  of  radar  navigation  intermediate  test,  Plj-A  1 


School 

Part 

N 

Median  dlin- 
culty  upper 
50  percent  1 

Median  diffi¬ 
culty  lower 
50  pcrcMit  * 

1  . 

I . 

1,050 

620 

87 

70 

2 . 

I . 

CI 

49 

1 . . . 

II  . . . 

83 

72 

o 

II  . 

77 

52 

1 . 

III.... . 

GS 

01 

2 . 

Ill . 

520 

03 

49 

M  cxl  lan 
ph!  * 


0.  IB 
.11 
.  10 
.  14 
.  13 
.11 


>  DiUn  from  school  1,  Vfctorvlllo,  nrc  from  classes  45-12  through  45-27.  Dnta  from  school  2,  Boco  Raton, 
aro  from  rla:-:cs  45  7  through  545. 

»  lla’i'i  1  upon  the  number  nttoinptlng  tho  Item.  . 

» ll.i.ed  upon  tho  upper  50  percent  versus  tho  lower  60  percent. 


Iludnr  Bombing  Intermediate  Test 

Tho  radar  bombing  intermediate  test8  consists  both  of  technical 
knowledge  and  problem-solving  items  measuring  knowledge  of  the 
theory  and  techniques  of  radar  bombing.  Tho  various  forms  of  the 
test,  which  include  from  52  to  77  multiple-choice  items,  aro  not  speeded 
and  require  somewhat  less  than  an  hour  to  complete.  A  radar  bomb¬ 
ing  intermediate  test  was  administered  to  radar  observer  students  near 
the  end  of  their  training  after  the  lectures  covering  radar  bombing  had 
been  completed. 

The  radar  bombing  test  was  developed  entirely  by  the  Radar  Proj¬ 
ect,  its  only  antecedents  being  daily  quizzes  based  upon  bombing  lec¬ 
tures.  Item  material  was  also  garnered  from  bombing  manuals,  lec¬ 
tures,  and  recommendations  of  instructors  and  supervisors.  Items 
were  di  tributed  randomly  with  respect  to  subject  matter  coverage 
which  included  such  topics  as  formation  bombing,  ofl-net  bombing,  co¬ 
ordinated  bombing  procedures,  set  operation  during  the  bombing  run, 
drift  determination  for  eolhsion  course  to  target,  and  procedure  turns. 

All  forms  of  tho  test  except  one  were  without  part  divisions;  tho 
exception  was  a  two-part  test  developed  for  use  in  tho  Western  Flying 
Training  Command  schools.  Tho  problem-solving  items  were  con¬ 
structed  in  interdependent  groups.  Caro  was  taken  in  selecting 
mislead  values  for  tho  problem-solving  items  to  make  them  close 
enough  to  tho  correct  value  to  reduce  to  a  minimum  the  likelihood  that 
a  student  could  do  careless  work  and  still  select  the  right  answer. 


•Developed  by  Sgt.  Groff,  Sgt.  Krlcdt,  nnd  Cpl,  Kocb. 
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Sample:  Interrelated  problem-solving  items.  Questions  1-0  nro 


parts  of  the  same  problem. 

Given : 

True  course  to  I.  P -  085°, 

True  course  from  I.  P.  to  target - - -  035°. 

True  air  speed -  170  K. 

Wind _  230°/ *0  K 

Absolute  altitude -  24,320  feet. 


1.  Find:  Radius  of  turn. 

1-A  3.0. 

1-B  3.2. 

1-C  3.4. 

1-D  3.6. 

1- E  3.8. 

2.  Find :  True  heading  to  I.  P. 

2- A  093°. 

2-B  095°. 

2-C  097°. 

2-D  099°. 

2- E  101°. 

3.  Find :  True  heading  from  I.  P.  to  target 

3- A  024°.  . 

3-B  026°. 

3-0  028°. 

3-D  030°. 

3- E  032°. 

4.  Find :  Wind  correction  vector. 

4- A  1.2. 

4-B  1.4. 

4-C  1.6. 

4-D  1.8. 

4- E  2.0. 

5.  Find:  Turn  allowance  (horizontal  range). 

5- A  1.9  n.  m. 

5-B  2.1  n.  m. 

5-C  2.3  n.  m. 

5-D  2.5  n.  m. 

5- E  2.7  n.  m. 

6.  Find:  Turn  allowance  (slant  range). 

G-A  4.1  n.  m. 

6- B  4.3  n.  m. 

6-C  4.5  n.  m. 

0-D  4.7  n.  m. 

6-E  4.9  n.  m. 
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Technical  information  item. 

25.  With  azimuth  stabilization  OX,  the  target  drifts  to  (lie.  left. 
The  radar  observer  should  corn-el  the  pilot  to  the — 
it 5 -A  right  and  move  the  track  line  to  the  right. 

25-B  ri<dit  and  move  the  track  line  to  the  left. 

n 

25-0  left  and  move  the  track  line  to  the  right. 

25-I>  left,  and  move  the  track  line  to  the  left. 

25-0  left  and  not  move  the  track  line. 

Original  forms  of  the  radar  bombing  intermediate  tests  included 
Pie- A  for  the  AX/APS-lf),  Plf-A  for  the  AN/APS-15A,  and 
Plg-A  for  the  AN/APQ-ld.  Curricular  revisions,  local  school  re¬ 
quests,  and  the  results  of  item  analyses  created  the  need  for  several 
modifications.  Form  Ple-B  was  a  revision  for  the  AN/A  PS-15. 
Plg-A,  applicable  to  the  AN/APQ-13  set  which  was  used  in  the 
Western  Training  Command  schools,  was  revised  when  the  E-6B  com¬ 
puter  solution  of  procedure  turns  was  introduced  into  the  curriculum 
and  given  considerable  emphasis.  The  resulting  test  had  three  alter- 
,  nate  or  revised  forms  for  use  at  Victorville,  V  Plg-A,  V  Plg-B,  and 
V  Plg-C,  and  W  Plg-A,  for  use  at  Williams  Field.  The  problems 
in  this  test  required  the  inclusion  of  two  tables  giving  data  for  bomb¬ 
ing  computations. 

The  means  and  standard  deviations  for  three  forms  of  the  radar 
bombing  intermediate  test,  each  given  at  one  of  the  three  largest  train¬ 
ing  schools,  are  presented  in  table  5.6. 

Taui.k  5.0. — Means  and  standard  deviations  of  distributions  of  raw  scores  from 
various  forms  of  the  radar  bombing  intermediate  test 


Form 

Numher 

Items 

Tlmo 

N 

Mean 

Ple-B  . 

55 

Minute  i 
40 

S20 

41.03 

}*!»•- A  .  .  . 

57 

•10 

508 

31.21 

vrig-n . 

05 

40 

200 

40.88 

The  results  of  item  analyses  of  difficulty  level  and  internal  consist¬ 
ency  for  two  forms  of  the  radar  bombing  intermediate  test  are  pre¬ 
sented  in  table  5.7.  The  difficulty  level  statistics  indicate  that  the 


Taiuj:  5.7. — Median  difficulty  level  for  upper  and  lower  scoring  groups  and 
median  phis  for  two  forms  of  radar  bombing  intermediate  lest 1 


School 


Median  <1I(T1-  >Tr«l Inn  fl  1  in- 
cultv  upper  cully  lower 
50  iK'n  cnt  50  percent 


Median 
phi » 


1  Data  from  school  1,  V lei orville,  are  from  claws  -15 -  1G  through  45-27.  Data  from  school  2,  Boca  Baton, 
ore  from  dav-es  15  .5  thr-mcli  i3.5.  ihitu  from  school  3,  Uaneley  Held,  arc  from  classes  45—1 1  through  ,531. 

>  Based  ii’kjh  the  upper  50  percent  versus  the  lower  50  percent. 
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itmi.i  easy  since  a  high  percentage  of  students  answered  the 

ill  j i is  correctly. 

'i  h:'  odd-even  reliability  computed  from  ISO  cases  for  form  Ple-B 
is  0.12  which  corrects  by  the  Spearman-Brown  formula  to  0.59.  This 
fmia  has  55  items  and  a  time  limit  of  -10  minutes.  The  mean  and 
Mandurd  deviation  of  the  odd  scores  are  20.12  and  2.5G,  respectively. 
The  mean  and  standard  deviation  of  the  even  scores  are  21. 8G  and 
•2.81,  respectively. 

The  reliability  of  0.59  is  lower  than  an  odd-even  reliability  of  0.75 
found  for  part  B  of  Final  Test  II,  Plb-A,  which  consists  mainly  of 
radar  bombing  items  similar  to  those  in  the  intermediate  test.8  A 
value  of  0.G9  is  shown  in  the  same  table  for  a  similar  section  of  a  later 
test,  part  B  of  Final  Test  II,  Plb-B.  The  lower  reliability  of  the 
intermediate  test  may  be  partially  explained  by  its  shorter  length. 

Kadar  Intelligence  Intermediate  Test 

The  radar  intelligence  intermediate  test 10  consists  wholly  of  tech¬ 
nical  information  items  measuring  verbal  knowledge  of  scope  inter¬ 
pretation,  radar  countermeasures,  scope  photography,  current  the¬ 
atres  of  operation  and  radar  targets,  escape  and  survival  techniques, 
and  mission  briefing  and  interrogation.  The  various  forms  contain 
from  83  to  120  multiple-choice  items  and  require  from  75  to  85  minutes 
for  administration.  The  test  was  usually  administered  near  the  end 
of  the  course. 

The  radar  intelligence  tests  were  constructed  wholly  on  the  basis 
of  lecture  material  and  classroom  quizzes  air  :ady  in  use.  Three  radar 
intelligence  tests  were  developed  before  rapidity  of  change  from  ono 
area  of  bombing  operations  to  another  forced  the  abandonment  of  this 
test  in  the  proficiency  battery.  Pln-A  and  PIn-B  were  developed 
for  the  schools  using  the  AN/APS-15  and  AX/A  PS-15  A  sets  and 
Pip  A  for  the  AX/APQ-13  schools. 

Means  and  standard  deviations  obtained  for  each  class  taking  the 
various  forms  of  the  radar  intelligence  intermediate  test  are  not  pre- 
'  rated  because  the  constantly  changing  subject  matter  render  the 
re  nits  meaningless.  After  several  administrations  of  a  given  form, 
the  particular  subject  matter  tested  was  often  found  to  have  been 
eh  linaled  from  the  radar  intelligence  curriculum. 

One  phase  of  the  radar  intelligence  curriculum  for  which  no  ado- 
qn.ue  proficiency  measures  were  developed  is  scope  interpretation. 
Krpcatedly,  aerial  instructors  and  students  found  that  errors  in  radar 
bombing  accuracy  were  duo  largely  to  failure  to  correctly  interpret 
the  returns  on  the  PIT  scope.  However,  the  complexity  of  the  returns 
presented  on  the  scope  and  particularly  their  movement  and  increasing 

Uiblo  S.12  for  reliability  utatlstlcs  of  Final  Test  II.  rib- A. 

”  rtirse  testa  were  prepared  by  Syt.  Graft  with  tbe  Biipervlulon  and  old  of  SkC 

Krledt. 
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complexity  on  the  bombing  nm,  appear  to  limit  the  extent  to  which 
printed  tests  can  measure  scope  interpretation.  It  is  probable  that 
aerial  and  supersonic  cheek  items  can  be  developed  to  measuic  this 
important  function  more  adequately.  An  additional  testing  tech¬ 
nique  which  promises  to  yield  more  adequate  measures  of  scope  in¬ 
terpretation  ability  is  tbc  motion  picture  test. 

Final  Test  1 11 

Final  Test  I  is  a  radar  navigation  test  consisting  wholly  of  speeded 
problem-solving  items.  It  measures  proficiency  in  use  of  the  E-6B 
computer,  the  Weems  plotter,  and  the  dividers,  with  emphasis  upon 
air  plot  wind  determination,  and  navigation  on  the  basis  of  ground 
positions  obtained  from  photographs  of  the  PPI  scope.  Tho  test 
consists  of  4  parts  with  a  total  of  85  or  105  items,  depending  upon 
the  form,  and  requires  145  minutes  for  administration.  It  is  the 
first  of  two  test  booklets  which  comprise  tho  final  examination  given 
to  students  upon  completion  of  radar  observer  training. 

The  chief  source  of  items  in  Final  Test  I  was  sections  5  and  6  of 
the  Radar  Operator  Achievement  examination,  prepared  by  tho  Na¬ 
tional  Defense  Research  Committee  and  used  by  tho  Aircrew  Evalua¬ 
tion  and  Research  Detachment  No.  1  in  the  Eighth  Air  Force.12  The 
AERD  No.  1  revision  of  section  5,  dealing  with  position  and  direction 
of  flight,  was  used  in  part  B  of  Final  Test  I,  in  which  tho  student 
determines  ground  position  from  simulated  scopo  photographs.  Sec¬ 
tion  G  of  tho  NDRC  tests,  a  navigation  problem,  was  revised  by  AERD 
No.  1  and  became  the  simulated  mission  presented  in  part  A  of  Final 
Test  II.  Tho  revisions  carried  out  by  AERD  No.  1  wero  toward  closer 
simulation  of  tho  materials  used  in  actual  radar  navigation  and  con¬ 
sisted  of  employing  an  actual  combat  zono  mercator  map  and  simu¬ 
lated  scope  photographs,  tho  latter  being  more  realistic  than  tho 
NDRC  mimeograph  presentation  of  the  PPI  scopo.  Tho  motiva¬ 
tional  effects  of  these  devices  wero  apparently  great,  sinco  students 
taking  the  tost  seemed  to  feel  that  they  wero  practicing  a  mission 
they  might  fly  some  day  over  Germany,  In  its  revision  of  tho  AERD 
No.  1  version,  the  radar  project  supplied  misleads  for  tho  mission 
and  scope  plotting  sections  by  the  panel  system  described  on  page  61. 
Several  of  the  test  photographs  wero  improved  and  tho  E-6B  com¬ 
puter  and  nirplot  sections  wero  edited  and  expanded.  Tho  simulated 
mission  was  placed  first  in  the  test  booklet  to  take  advantage  of  tho 
greater  student  interest  in  this  section. 

Part  A  has  25  items  in  form  Pla-A  and  45  items  in  form  Pla-B. 
Both  forms  require  TO  minutes  to  administer.  Part  A  consists  of 
a  simulated  radar  bombing  mission  over  Germany,  the  separato  items 

11  Developmental  work  on  tills  test  wns  conducted  by:  Cnpt.  Horace  R.  Van  Saun,  Sgt. 
GrnfT,  S)jt.  Philip  11.  Krledt,  and  Cpl.  John  V.  MucNaughton. 

11  See  chapter  10. 
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of  which  call  for  solution  of  navigation  problems  that  would  typically 
bo,  met  on  such  a  mission.  The  student  is  provided  with  an  cxpend- 
a I,),,  mol  cal  or  map  of  the  area  over  which  the  bombing  mission  is 
(Pwn  a!  1  a  book  of  10  simulated  scope  photographs  representing 
(be  sc< i j  •'  at.  given  times  during  the  mission.  Using  the  E-GB  com- 
patir.  Weems  plotter,  and  dividers,  the  student  is  required  to  de- 
tennine  ground  position  from  the  photographs,  compute  winds,  de¬ 
termine  new  headings  to  make  good  a  given  course,  compute  estimated 
times  of  arrival  at  various  points,  and  solve  similar  basic  navigation 
problems.  The  speeded  items,  in  conformity  with  usual  test  construc¬ 
tion  practice,  increase  in  difficulty  through  the  test  and  are  necessarily 
interrelated  as  is  evident  in  the  following  item: 

Sample:  A  problem-solving  item  related  to  an  earlier  item. 

10.  In  order  to  avoid  flak  around  Munster  you  alter  course  and 
fly  directly  from  the  last  ground  position  (questions  13  and  14)  to 
52°  10'  N,  07°50'  E.  Using  the  wind  in  questions  17  and  18,  which 
is  closest  to  the  wind  that  you  actually  computed  and  the  same  truo 
air  speed  of  215  K,  find  the  new  true  heading. 

19-A  052°. 

19-B  056°. 

19-C  0G0°. 

19-D  164°. 

19-E  068°. 

The  responses  to  many  items  in  form  Pla-A  contained  two  inde¬ 
pendent  variables.  In  form  Pla-B  such  items  were  split  into  two, 
each  having  single-variable  responses.  This  change,  which  amounted 
to  increasing  the  number  of  items  in  part  A,  was  most  commonly  made 
in  items  requiring  wind  information  and  coordinate  readings. 

Sample :  Old  form: 

5.  Photo  No.  5  shows  tho  screen  at  0957.  What  is  the  wind? 

5 -A  345°/50  K. 

5-B  350V44  K. 

5-0  3 GO 757  K. 

5  1)  005  742  K. 

5-E  010°/G4  K. 

Sample:  New  form: 

7.  Photo  No.  2  shows  tho  sco^o  at  0957.  What  is  tho  direction  of 
the  wind? 

7- A  165°. 

7-B  175°. 

7-0  335°. 

7-D  345°. 

7-0  355°. 
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8.  AVliat  is  (lie  wind  force? 


S-A  10  K. 

8-B  10  K. 

8-C  57  K. 

8-D  G7  K. 

8-E  77  IC. 

Tolerances  were  determined  b\'  a  panel  of  in -t  ructors  and  by  test 
ronstrm  ’.ion  teams.  Some  of  the  considerations  were  accuracy  of  plot¬ 
ting  and  of  use  of  computing  equipment.  Included  in  alternative 
choices  were  answers  based  on  reciprocal  plots,  careless  settings  and 
inaccurate,  readings  of  the  E  -GB  computer,  and  common  misunder¬ 
standing  of  required  technique.  It  was  believed  that  the  .simulated 
mission  requires  much  the  same  selection  and  integration  of  data  that 
is  required  by  actual  radar  navigation.  The  simulated  mission  is  to 
be  contrasted  with  the  remaining  parts  of  Final  Test  I,  which,  like 
the  usual  test,  require  solution  of  independent  items. 

Part  B  consists  of  20  items  and  has  a  time  limit  of  30  minutes.  Each 
item  requires  the  student  to  determine  ground  position  from  a  simu¬ 
lated  scope  photograph.  The  radar  returns  appearing  on  the  20 
photographs  represent  areas  on  the  Mercator  map  used  for  part  A. 
A  dead-reckoning  position  is  given  with  each  photograph  which  is 
within  approximately  50  miles  of  the  represented  ground  position. 
The  student  must  identify  one  or  more  returns  on  the  photograph,  take 
a  fix  on  these  returns,  and  plot  it  accurately  on  the  map.  The  answer 
is  recorded  by  selecting  the  one  of  five  pairs  of  latitude  and  longitude 
values  that  corresponds  most  closely  to  the  coordinates  of  the  plotted 
point.  Selection  of  the  correct  pair  by  elimination  was  made  unprofit¬ 
able  by  the  enforcement  of  a  severe  time  limit.  Several  conditions 
govern  the  selection  of  misleads.  Mislead  values  were  chosen  which 
were  close  enough  to  the  correct  values  to  measure  plotting  accuracy. 
At  the  same  time,  however,  misleads  were  given  values  such  that  rea¬ 
sonable  plotting  deviations  would  not  force  a  student  to  select  at 
random  one  of  two  choices  equidistant  from  his  plotted  point.  An¬ 
other  con  ideration  was  that  a  degree  of  latitude,  as  represented  on 
l lie  Mercator  map,  was  00  percent  longer  than  a  degree  of  longitude. 
Because  of  this  scale  difference,  a  vertical  plotting  error  of,  for  ex¬ 
ample,  2  minutes  represented  poorer  performance  than  a  horizontal 
error  of  2  minutes.  On  this  basis,  for  each  item,  the  longitude  values, 
in  the  misleads  differed  from  the  true  longitude  value  slightly  more 
than  the  mislead  latitude  values  differed  from  the  true  latitude. 

Sample:  Scope  plotting,  part  B  (Each  question  is  supplemented 
with  a  scope  photograph.) 

•1G.  Approximate  position  at  time  of  photo  46: 

51°  10'  N,  0G°  20'  E. 
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Kind  .  ( !  round  {>;>.  ition  at  t  ime  of  photo  16. 

•it;  A  r»  1  ° i:i'  X.,  or>°2!'  K. 
to  b  i  'j;V  x.,  or»°:u)'  E. 

I0-( 5  1 1  '.VI'  X.,  05 "40'  E. 

•to  ])  52^20'  X.,  07°00'  E. 

■10  E  52°30'  N.,  06° 50'  E. 

P.irt  C  consists  of  20  items  requiring  12  minutes  for  form  Plrr-A  and 
15  minutes  for  form  Pla-B.  Each  item  requires  the  student  to  use 
the  E-GB  computer  to  solve  a  navigation  problem,  such  as  finding  wind 
direction,  wind  force,  track,  heading,  and  ground  speed.  The  student 
selects  the  answer  to  each  item  from  among  five  alternatives. 

In  setting  up  mislead  alternatives,  attempts  were  made  to  rule  out, 
as  far  as  possible,  the  chance  selection  of  the  correct  answer.  Several 
of  the  four  misleads  were  given  values  near  the  correct  value  in  a 
number  of  the  items.  The  problem  then  arose  of  the  size  of  errors  that 
were  due  to  variation  in  the  accuracy  of  the  E-GB  computers.  Guid¬ 
ance  on  this  problem,  was  found  in  an  unpublished  study  conducted  in 
January  19-15  by  the  Navigator  Project.  In  order  to  determine  the 
magnitude  of  computer  differences,  the  Navigator  Project  used  eight 
each  of  four  common  types  of  computer  to  solve  a  number  of  typical 
navigation  problems.  The  distribution  of  the  answers  about  the 
arithmetic  mean  of  all  solutions  indicated  that  computer  variability 
was  negligible.  Only  7  percent  of  the  computers  had  errors  greater 
than  1  knot  for  the  computation  of  true  air  speed  from  indicated  air 
speed,  pressure  altitude,  and  temperature.  Drift  determination,  using 
a  10-knot  wind  45°  from  the  true  heading,  produced  nil  error  of  i/o0  or 
greater  in  only  3  percent  of  the  cases.  Ground  speed  in  the  same 
problem  was  in  error  by  1  knot  or  more  for  only  4  percent  of  the 
computers. 

Misleads  for  the  E-GB  computer  problems  also  incorporated  com¬ 
mon  errors  in  procedure.  For  example,  when  problems  involve  high 
velocity  winds,  it  is  necessary  to  enter  the  wind  velocity  on  the  com¬ 
puter  at  a  fraction  of  its  value.  At  the  same  time,  the  true  air  speed 
nuisl  ho  entered  at  the  same  fraction  of  its  value  and  the  resulting 
ground  speed  must  be  multiplied  by  the  reciprocal  of  the  fraction.  In 
•‘-'H-.li  problems,  the  values  that  would  be  obtained  if  a  student  over¬ 
looked  ono  or  more  of  the  enumerated  steps  were  given  as  misleads. 

Sample:  E-GB  computer  problems. 

74.  Given: 

Wind. _ 

True  heading-. 

True  air  speed 
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0S0°/70  K 
075° 

250  K 


Find  :  Ground  speed  : 

74- A  148  K. 

74-B  180  K. 

74 -C  1S8  IC. 

74 -D  205  K. 

74-K  208  K. 

Purl  D  consists  of  20  airplot  problems  with  u  time  limit  of  30  min¬ 
utes.  The  problems  require  the  use  of  the  E-OB  computer,  Weems 
plotter,  and  dividers  in  determining  winds  by  the  airplot  method  or 
carrying  out  dead-reckoning  procedures.  The  plotting  is  done  on 
the  unused  portions  of  the  mcrcator  map  used  for  parts  A  and  B.  The 
student  is  required  specifically  to  determine  air  position,  wind  velocity 
and  direction,  and  estimated  time  of  arrival  at  a  proposed  point  in  the 
(light.  Several  groups  of  interrelated  items  are  included.  The  mis¬ 
lead  alternatives  for  the  five  choice  items  were  obtained  from  the  com¬ 
putations  of  the  panel  of  experts  and  from  a  consideration  of  common 
errors  in  procedure.  Consideration  was  given  to  the  difference  in  lati¬ 
tude  and  longitude  scale,  and  to  the  greater  inaccuarcy  of  short  com¬ 
pared  to  long  wind  legs. 

Sample :  Airplot. 

Questions  88  and  89  are  parts  of  the  same  problem. 

88-89  Given: 

Point  of  departure _  53°32'  N.,  05°38'  E. 

Time  of  departure _  0829. 

True  heading _ : _ ' -  060°. 

True  air  speed _ _ _  192  K. 

Time  of  turn  and  PPI  fix _  0843. 

Coordinates  of  PPI  fix _ _  54°04'  N.,  06°42'  E. 

New  true  heading _  210°. 

New  true  air  speed _  198  K. 

S3.  Find:  Wind. 

83- A  160750  K. 

88 - B  108712  K. 

S8-C  108743  K. 

S3-D  177712  K. 

8 3 -JO  177743  K. 

89.  F: ml :  Bi  4.  estimate  of  ground  position  at  0850.  (Use  tho  wind 
in  question  88  that  is  closest  to  the  one  you  actually  computed.) 

89 - A  53°3F  N.,  00°09'  E. 

89-B  53°3(i'N.,0G°0G'  E. 

89-C  53°39'  N.,  0G°00'  E.  . 

89-D53o40'N.,0G°10'  E. 

89-E  53°44'  N.,  06°10'  E. 
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Fin  1 1  Test  I  is  applicable  to  curricula  built  around  eilher  tho 
\N7ATS-15,  AiST/APS-l.r)A,  or  AN/Al’Q  Id  sols  since*  it  is  concerned 
,  rimurily  with  elementary  navigational  techniques.  Two  forms  of 
Final  Test  I  were  prepared,  Pla-A  and  Pla-B.  ,  Pla  B  differed 
from  the.  earlier  form  primarily  in  that  the  simulated  mission  was 
expanded  from  25  to  45  items  without  an  increase  in  testing  time, 
'[lie  process  by  which  this  was  accomplished  as  mentioned  earlier, 
consisted  of  splitting  items  with  two-variable  answers  into  two  items, 
each  having  one- variable  answers.  Also  complex  problems  were 
broken  down  into  intermediate  steps  and  an  answer  was  required  for 
each  step.  The  items  in  Pla-A  were  analyzed  and  items  were  deleted 
on  the  basis  of  lack  of  relation  with  total  test  score  or  because  of 
extremely  high  or  low  difficulty  level.  The  items  in  the  F.-GB  and 
aii plot  sections  were  re-arranged  and  the  time  limit  for  the  E-OB 
section  was  shortened  from  15  to  12  minutes  to  reduce  the  number  of 
students  completing  the  section.  The  latter  change  was  necessary  to 
increaso  the  discriminatory  power  of  this  test  at  the  upper  levels  of 
proficiency. 

Tho  means  and  standard  deviations  for  the  part  and  total  scores  of 
both  forms  of  Final  Test  I  are  presented  in  table  5.8.IS 


Tabus  5.8. — Means  and  standard  deviations  of  distributions  of  part  and  total 
raw  scores  from  two  forms  of  final  test  l 1 


Part 

Form 

Number 

Items 

Tlmo 

N 

Mean 

8D 

A . 

Pla-A . 

25 

Mlnuiet 

70 

236 

15.90 

4.00 

Pla-B  . 

45 

70 

085 

7.51 

B . 

Pla-A . 

20 

30 

230 

9.41 

3.29 

Pla-B . 

20 

30 

685 

10.44 

3.72 

C . 

Pla-A  . 

20 

15 

230 

3. 07 

PIn-B . 

20 

12 

085 

13.01 

3.77 

D . 

Pla- A  . 

20 

30 

230 

8. 14 

2.41 

ria-B . 

20 

30 

635 

9.39 

3.12 

Total . 

Pla-A  . 

85 

115 

230 

40.72 

10. 25 

Pla-B  . 

105 

142 

035 

63.93 

12.62 

*  Hat  a  for  Form  Pin- A  nro  based  upon  Langioy  Field  classes  4-1-12  through  44-16  and  for  Form  Pla-B, 
l.-.tyley  l-’icld  clos  e3  15-8  through  45-24. 


In  table  5.9  are  presented  the  means  and  standard  deviations  for 
the  part  and  total  scores  of  form  Pla-B  for  three  schools. u  It  will 
he  noticed  that  most  of  the  critical  ratios  of  the  differences  between 
die  means  are  significant  at  the  1  percent  level.  The  differences  arc  ex¬ 
plainable  in  terms  of  population  differences  and  tho  varying  emphasis 
upon  navigation  procedures.  Students  from  both  the  highest  and  low- 

’AU  statistical  work  was  conducted  under  the  supervision  of  S/Sgt.  Bernard  C.  Sullivan. 
He  vkAs  assisted  by  Cpl.  ltobert  H.  Koch,  Sgt.  Samuel  D.  Morford,  and  Cpl.  Sofer.  For  Final 
1,'d  11  and  subsequent  testa,  Cpl.  Koch  discontinued  work  with  statistical  analysis  and 
contributed  directly  to  test  construction  with  Item  development. 

11  Critical  ratios  were  computed  by  Cpl.  Sofer,  and  checked  by  Cpl.  Wilbert  U. 
Scliwotzcr. 
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T.mii.k  Com pn/'i'-'on  of  st-hnnls  iii  hrnii  of  inriinx  and  standard  deviations 

of  distributions  of  rai  l  and  total  nut;  scons  of  Final  Test  I,  I‘la  It  1 


Critical  ratio  of  in. -an  dill' 

niKv.s 

I’nrt 

School 

N 

Mean 

-  -- 

j 

— 

SI) 

1 

2 

3 

1 

»>5 

2‘)  M 

7  ^ n 

14.21  1 

4  2J 

2 

4% 

2H.  02 

7.  tin 

H  21 

12 

3 

0  25 

20.  ;:t 

7.  18 

4  22 

12  30 

Total 

1. 

*JI..  I>  1 

7.M 

I 

r.'-'i 

10.  II 

3.  72  i 

:>  \u\ 

0.  (25 

2 

490 

s.  :ts 

3.  58 

9.  03 

3.  27 

3 

rrr> 

9  CO 

3.  lit 

0.  05 

3.  27 

Total  .... 

1, 

0.  43  I 

3.  00 

' 

J 

tw/i 

n.  in 

3  ,7 

-  — 

in.  :,i 

1  73 

2 

-100 

10.02 

3. 02 

Hi.  5-1 

1  -i  2 

3 

Clio 

1  \  27 

3.  33 

1.73 

15  40 

. 

Total  ... 

1 .  SIX  * 

1 2.  r»o 

3.00 

1 

).. 

I 

r.s.s 

9.39 

3.  12 

: - -■ 

13.  49 

4.  61 

2 

4% 

0.  32 

2.  lit) 

1H.  It) 

14.02 

3 

025 

8.  02 

2.  S3 

4.01 

_ 

14.02 

Total  . 

1,80  A 

8.  37 

2.00 

Total  . 

l 

63.  03 

12.  52 

21.  00 

8.  SO 

2 

400 

45. 50 

13.  20 

21  00 

15.20 

3 

025 

57.  (17 

13.  11 

8.80 

15.20 

Total . 

1,800 

57.  19 

1194 

1  H;il a  from  school  1,  I,  inpley  Kidd,  an  from  classes  45  SlhroiiRh  45-2-1.  Dot u  from  scho<  1  2,  Horn  Raton 
arc  from  classes  11  0  throiipti  11-125.  Data  from  school  3,  Victorville,  arc  from  classes  I5*j  through  45-21 


c.st  scoring  schools,  No.  1  and  No.  2  respectively,  were  predominantly 
bombardiers.  This  tends  to  emphasize  the  efYect  created  by  curricular 
differences. 

No  reliability  statistics  are  available  for  this  test  since  all  parts 
v.cro  speeded  and  no  alternate  form  was  available  to  use  in  determin¬ 
ing  a  tcst-rctest  reliability. 

Difliciilty  level  and  internal  consistency  statistics  for  form  Pla-B- 
from  administrations  to  Langlev  Field  classes  45-8  through  45-30  are 
given  in  table  5.10.  It  will  be  noted  that  the  airplot  section,  part  D, 
is  the  most  difficult,  followed  by  the  ground  position  section  and  the 
simulated  mission,  with  the  E-GB  computer  section  being  the  easiest. 

T.vnu;  5.10. — Median  difficulty  level  for  upper  and  lower  scoring  groups  and 
median  phis  for  the  parts  of  Final  Test  J,  Pla-Bx 


Tart 

N 

•Median 
dillicnlly 
upper  50 
percent  * 

M  ed  Ian 
<1  hllculty 
lower  50 
percent 1 

Median 
phi  • 

A . . 

800 

88 

71 

0.22- 

a .  . 

Kt>G 

78 

50 

.25 

o . 

800 

91 

71 

.23 

n  . . 

803 

77 

51 

.22 

>  lla-vd  on  I.angley  Fldil  classes  45-3  through  45-30. 

1  11. i  I  upon  llio  number  nltomiHlni:  tho  Item. 

•  IIilsviI  upon  the  upper  10  percent  versus  the  lower  50  percent. 
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<liHi(-nll,y  figure's  aro,  of  course,  bused  upon  the  number  at^Mipt- 
ju,r  [he  item.  The  interrebit inns  of  the  four  purls  of  Final  Tot  I  aro 
pn  ciitod  in  chapter  8. 

final  Test  II. — Final  Test  II 15  measures  knowledge  of  the  air-borne 
r;n la r  set,  radar  bombing,  and  radar  navigation,  The  various  forms 
„f  i v"'t  include  from  80  to  110  technical  information  items  and  aro 
admiiii'-teied  in  a  1-hour  testing  period.  None  of  the  forms  is  speeded. 
Final  Test  II  is  the  second  of  two  booklets  which  comprise  the  linal 
examination  given  to  students  upon  completing  the  course. 

As  was  true  of  Final  Test  I,  the  items  for  Final  Test  II  were  ob¬ 
tained  primarily  from  items  developed  by  the  Air-crew  Evaluation 
and  Research  Detachment  No.  1  which  were,  in  turn,  suggested  by  ma¬ 
terial  in  the  National  Defense  Research  Committee  Operator  Achieve¬ 
ment  Examination.  New  items  were  constructed  from  course  quizzes, 
lectures,  and  training  manuals. 

Most  of  the  forms  of  Final  Test  II  contained  two  parts:  the  first 
was  concerned  with  knowledge  of  set  operation  and  paiticnlarly  the 
functions  of  the  components  of  the  set  used  on  a  radar  bombing  ran; 
the  second  covered  the  theory  and  procedures  of  radar  bombing  and 
navigation  with  the  emphasis  upon  bombing.  Questions  involving 
the  student’s  knowledge  of  the  appearance  of  specific  targets  on  the 
PPI  scope  were  also  included  in  the  second  section.  1 

The  main  consideration  that  guided  the  construction  of  misleads 
for  the  five  alternative  items  was  that  of  plausibility.  Many  of  these 
incorrect  choices  included  typical  student  errors  arising  from  lack 
of  understanding  of  required  techniques.  Sample  items  concerned 
with  navigation  and  set  operation  are  given  below. 

Sample  item:  Navigation  technique. 

‘27.  Azimuth  stabilization  is  ON.  At  the  IP  the  pilot  has  been 
given  a  true  heading  of  170°  to  make  good  a  true  course  of  173°. 
After  the  turn  is  made  the  target  appears  at  17G°.  What  is  the  proper 
correction? 

27-A  G°  right. 

27-B  3°  right. 

•27-C  3°  left. 

27-D  G°  left. 

27-E  9°  left. 

Sample  item  :  Set  operation  (technical  information)  : 

93.  The  spinner  should  not  be  stopped  pointing  directly  toward 
beacon  because 

93- A  beacon  will  stop  transmitting. 

93-B  beacon  will  overload  and  cut  out  other  aircraft* 


15  This  tent  wnn  developed  under  (lip  supervision  of  Capt.  Van  Snun.  Tbe  chief  con- 
tribu tors  were:  Sgt.  Graff,  Sgt.  KreUlt,  and  Cpl.  Koch. 
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93 -C  the  radar  .sot  will  not  pick  up  any  signal. 

93- D  o  ill  or  aircraft  will  pick  up  your  signal. 

93 -K  all  beacon  signals  will  fuse. 

Time  limits  were  assigned  to  the  test  which  allowed  most  students 
to  complete  all  items.  For  the  two-part  forms,  time  limit  markers 
were  placed  through  (lie  tost  to  indicate  to  the  student  the  adequacy  of 
his  working  rate. 

Four  two-part  forms  of  Final  Test  II  were  constructed,  ]  ib-A  and 
Plb-B,  for  the  AN/APS-15,  and  Pld-A  and  Pld-B  for  the 
AM/APQ-13.  An  additional  four-part  form,  V  Pld-A  for  the 
AN/APQ-13,  was  prepared  only  for  the  radar  schools  of  the  Western 
Flying  Training  Command.  The  revisions  were  made  necessary  by 
fluctuations  in  the  content  of  the  curricula  which  varied  between 
schools  and  from  time  to  time  at  the  same  school.  For  example,  at  one 
school  lectures  on  oil-set  bombing  were  eliminated  from  the  curricu¬ 
lum  only  to  bo  reestablished  later.  V  Pld-A  was  made  necessary  by 
tho  western  schools’  emphasis  upon  target  timing,  wind  determina¬ 
tion,  off-set  bombing,  and  procedure  turns. 

The  means  and  standard  deviations  of  the  part  and  total  scores 
of  forms  Plb-A  and  Plb-33  for  classes  at  Langley  Field  are  given  in 
table  5.11.  These  classes  consisted  of  both  bombardiers  and  navi¬ 
gators  with  bombardiers  becoming  predominant  in  classes  45-10  to 
45-35. 


Table  5.11. — Means  and  standard  deviations  of  distributions  of  part  and  total 
raw  scores  for  two  forms  of  Final  Test  II1 


Tort 

Form 

Number 

Items 

Tlmo 

N 

Mean 

8D 

A  _ 1 

rib-A . 

50 

Afiriufcs 

27 

23(5 

11 

7. 01 

Plb-B . 

CO 

35 

c.so 

0.74 

D. 

Plb-A . 

50 

23 

23(5 

jlMgSIS 

6.51 

l‘lb-11 . 

40 

20 

580 

6.20 

Total . 

\mmi 

100 

55 

2.30 

CA  02 

1  mm 

■  .  H 

100 

55 

321 

71.00 

■HB 

1  for  form  Plh-A  are  from  Lanr.ley  l'I'  M  cla.-vs  41-12  llirouKli  14-10.  Data  for  port  rooms  of  form 
I'lb-ll  are  from  l.uiclcy  1  loll  c!  s  15-3  Ummi-li  15-24.  Data  for  tho  total  score  of  form  l’lb-D  aro  from 
Langley  Field  cl  .-.h  -10  3  through  45  12  and  45-22  through  45-24. 


In  table  5.12  are  given  the  results  of  reliability  studies  of  two  forms 
of  Final  Test  II.  Form  Plb-A  appears  to  have  a  reliability  of  about 
0.80  while  Plb-B,  given  to  later  classes,  seems  to  have  a  reliability  of 
approximately  0.70.  One  possible  explanation  for  the  apparent  lower 
reliability  of  the  second  form  is  that  the  students  were  believed  to  be 
more  homogeneous  in  ability  in  biter  classes.  As  will  bo  noted  from 
the  table,  the  standard  deviations  for  the  second  form  aro  low  as 
compared  to  those  for  tho  first, 
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'I'a'.h  k  r-.rJ.  Ottil-rrrn  and  l\  !rr  -Richardson  riUubility  coefficients  for  tico 

forms  of  final  Test  II 


ft  nil 


I’lb  A  1 
rib  n’  ■ 

Tot  ul 


Korin 


rib- A  ' 


Total. 


Part 

N 

A!  (gin- 
mill 

Mean- 
c\  i-n 

Sl>- 

.'•1.1 

Sl>. 

even 

rlu 

rn1 

A 

2ir> 

)  t  53 

. 

lt«  t>2 

1.  16 

3.30 

0.  73 

0  S4 

11 

m.  io 

IS.  53 

:t.  (3 

J.  7  f> 

.  (-0 

.  75 

A 

mo 

m  s.5 

•.’II.  Irll 

:i.  o: 

2.  hh 

.  43 

.  0.4 

11 

mu 

1  3.  S3 

15. 00 

2.  70 

2. 62 

.03 

.... 

iso 

33.  00 

35.  53 

1  S5 

4.  20 

R3 

__-7! 

Part 

t 

N 

Tol. i  mean 

Total  SO 

Mi  4 

A 

236 

31.2 

7.0 

0.73 

B 

2.36 

34.0 

5.5 

.67 

. 

230 

00.0 

1 

10.9 

.32 

i  Odd-even  cootl. ciont  corroded  by  the  Spearman-Brown  formula, 
i  li:,ra  foi  form  Plb-A  nro  from  l.anidey  KleM  classes  44-12  through  44-10. 
i  piii'i  for  form  i'll)- 11  nre  from  Bangley  Field  closscs45-I0  through  45-33. 
i  Kuder-Kielmrdson  reliability  coefficient. 


Item  diniculty  and  internal  consistency  data,  computed  for  Victor¬ 
ville  classes  1-5-12  through  45-23,  are  presented  in  tablo  5.13.  Al¬ 
though  this  test  is  more  difficult  than  Final  Test  I,  it  is  still  relatively 
easy,  having  all  average  item  difficulty  level  of  approximately  70 
percent  passing. 


Tatii.k  5.13. — Median  difficulty  level  for  upper  and  lower  scoring  groups  and 
median  phis  for  pr.r/s  of  Final  Test  II,  Pld-B 


Port 

N 

Median  dim- 

culty  upper 

50  percent 1 

Median 
phi  * 

A . 

GOO 

79 

60 

0. 12 

B.  . 

GOO 

73 

G6 

.10 

'  rm  d  upon  the  number  attempting  (lie  Item. 

1  I'..-,  i  d  upon  the  upper  50  percent  versus  tlio  lower  50  percent. 


F!n  il  fest  for  AN/APQ-7. — The  final  test  for  AN/APQ-7  consists 
of  technical  information  and  problem-solving  items  measuring  knowl¬ 
edge.  if  the  operation  and  functions  of  the  AN/APQ-7  set  and  its 
’•  "o  in  radar  bombing  and  navigation.  The  test  consists  of  120  items 
and  requires  07  minutes  to  administer. 

1  he  AX/APQ-7  is  an  air  borne  radar  set  that  is  especially  valuable 
is  bombing.  .Students  in  the  AN/APQ-7  course  had  already  grad- 
1 1  ' r  1 1  from  the  10-week  AN/APQ-13  course  and  were  given  4  weeks 
f)f  additional  training.  Only  one  school  in  the  radar  training  program 
Ought  the  use  of  this  equipment.  It  was  necessary  to  develop  a  spe- 
l’ud  final  examination  because  the  operating  procedures  and  functions 
or'  dm  AN/APQ-7  were  different  from  those  of  any  other  set  for 
v-hieh  the  Radar  Project  had  constructed  a  test, 

Ihe  it. -m  material  in  the  final  test  for  AN/APQ-7  was  compiled 
entirely  by  the  Radar  Project.  Test  construction  teams  participated 


in  ample  f  ruining  missions,  altendid  via  e*\  and  cxammcd  Pel  ir-cs 
and  I  ra ini tm  outlines.  As  ilcius  were  cmi  l  r acted.  1  hr  v  were.  dir  ii: ,  c  j 

n  -  •> 

with  iust  ruelm  s  in  tin*  AX/APQ  7  course. 

'rin*  final  test  for  AX/APQ -7  consists  of  live  sections.  The  fn.,1 
three  are  made  up  of  problem-solving  navigation  and  bombing  item  , 
while  the  final  two  mea-.me  verbal  knowledge  of  equipment,  and  pro¬ 
cedures.  Section  A  is  speeded  and  includes  20  problem  solving  item* 
given  with  a  time  limit  of  DO  minutes,  The  items  require  the  student, 
to  determine  wind  force  and  direction,  true  headings  for  bombing 
runs,  range-wind  factors  and  ground  speeds  to  set  into  the  AN/APQ-7 
computer  for  the  bombing  run,  and  drift  angles  to  set  into  the  bomb- 
sight.  Auxiliary  equipment  required  to  solve  these  problems  includes 
a  Mercator  projection,  an  E-GB  computer,  an  N-T  ground  speed  coin 
pnter,  a  Weems  plotter,  and  dividers.  The  technique  used  for  form 
Pla-B  of  Final  Test  I  of  splitting  wind  force  and  wind  velocity  into 
separate  items  was  employed  to  increase  the  number  of  items  without 
increasing  the  necessary  testing  time.  The  primary  consideration  in 
constructing  the  mislead  alternatives  for  the  five-choice  items  was  to 
penalize  students  for  inaccuracy  while  allowing  for  reasonable  devi¬ 
ations.  Several  sample  problems  are  shown  below. 


Sample :  1-1.  Given : 

Point  of  departure _ 

Time  of  departure _ 

True  heading _ 

True  air  speed - 

Time  of  GPI  fix _ 

Target  of  GPI  fix _ 

Bearing  range  of  GPI  fix 

Initial  point - 

Target _ 


.  42°19'  N.,  135°20'  E. 
.  0928 
.  135° 

.  210  m.  p.  h. 

.  0946 

.  41° 00'  N.,  13G°1C'  E. 
.  13°  11/30  S.  M. 

.  40°40'  N.,  136° 20'  E. 
.  10°20'  X.,  135°50'  E. 


1.  Find:  Wind  direction. 


1-A  050°. 
1-B  000°. 
1-C  075°. 
1  D  233°. 
1-E  245°. 


2.  Find  :  Wind  force. 

2-A  37  m.  p.  Ii. 

2-B  43  m.  p.  h. 

2  C  49  in.  p.  h. 

2-1)  55  in.  p.  Ii. 

2-E  G9  m.  p.  h. 


u  Developed  by  Sgt.  Gngo. 
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3.  Find :  True  heading  from  initial  point  to  target. 

3- A  036°. 

3-B  043°. 

3  C  050°. 

3-D  227°. 

3- E  237°. 

4.  Find :  Ground  speed  to  be  set  into  computer  for  bombing  run. 

4 - A  157  m.  p.  h. 

4-B  1G2  m.  p.  h. 

4-C  278  m.  p.  h. 

4-D  290  in.  p.  h. 

4-E  301  m.  p.  h. 

Section  B  is  a  speeded  section  which  includes  eight  items  with  a 
time  limit  of  15  minutes.  These  items  measure  proficiency  in  comput¬ 
ing  procedure  turns  and  require  the  use  of  an  E-6B  computer  and  three 
tables  besides  the  navigation  data  given  in  tho  item.  The  items  are 
multiple  choice  with  no  regularity  of  tolerances  between  choices.  A 
sample  item  is  given  below. 

Sample: 

True  head-  True  head •  Tiue  air  Absolute 

Wind  (statute  In g  to  tar-  ing  to  speed  altitude  Slant  range  from  IP  at  which  procedure 

miles)  gel  1.  P.  (m.p.h.)  (feet)  turn  should  be  started  (statute  miles) 

23.  1907-10  240°  180°  180  10, 000  A  B  C  D  E 

1.  4  2.1  2.3  2.9  3.3 

Section  C  is  a  speeded  section  of  24  items  with  a  time  limit  of  seven 
minutes.  The  items  require  the  student  to  solve  drift  problems  on  tho 
radar  bombing  run,  making  corrections  for  drift  and  determining 
amount  and  direction  of  drift  on  a  given  heading.  No  equipment  is 


required  and  answers  are 

multiple-choice.  A 

sample 

item 

of  this 

section  is  given  below. 
Sample: 

Distance 
of  first 
bearing 
(‘tatute 

Distance 
of  second 
bearing 
( statute 

First 

Second 

miles) 

mites) 

btariiig 

bearing 

31-33 . . 

30 

25 

4°  L. 

2°  L. 

Answer : 

31.  Direction  of  correction 

A 

L 

B 

R  . 

C 

D 

E 

32.  Amount  of  correction _ 

2° 

4° 

G° 

8° 

12* 

33.  lOecorrcction  drift . . 

...  14°  L. 

10°  L. 

8°  L. 

8°  R. 

10°  R, 

Section  D,  not  speeded,  consists  of  33  technical  information  items 
and  h  as  a  time  limit  of  20  minutes.  The  items  measure  verbal  knowl¬ 
edge  of  equipment  function  and  operation,  especially  emergency  cali¬ 
bration  procedures  and  location  of  burnt-out  fuses.  In  constructing 
misleads,  the  primary  concern  was  for  plausibility. 


703327 — 17- 
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i  tuple  : 

58.  S 

call  /'To 

•tin  t  Ill'll  .-Wi  -  !  ■ 

■  V 

h 

!  1  ■  til 

ju  ted 

so  that — 

58  -A 

the 

Wi  ep  |-  u 

f  till!  I'll  111  ia  * t y 

.  T  -'ll 

0  mile 

to  GO  miles. 

58  B 

the 

V.  1  1  •  p  1  >  • 

>  f  1 1 11 1  f  1  U  1 . .  .  .  '  i  v 

fl'  : 

o  ;0  ■ 

left  or 

■  right. 

th 

rough  0  . 

58  C 

t  lion 

■  is  a  pn 

■  mi i,i"*t  hr igh;  1  < ne  i 

low; 

1  (he  ' 

■  ntor 

of  the 

;  e 

ope. 

58-1) 

t  hel  i 

•  is  :■  lief! 

1  m  1 1  ■  d .  1 1  1\  1 1 '  i  e  d  n 

1  he 

i.ier 

of  the 

scope. 

58-K 

t  liel'i 

•  i  .a  thin 

bright  v.cl  re  dov.  : 

•lit 

•;.!  r 

of  the 

scope. 

et  t  ion  I 

•k  not 

v  .periled. 

In  v  '■)'•'>  item-  and  a  ; 

h,  ■  ' 

!  i '  1 1  i  t  v  i 

f  25  minutes. 

The 

items 

Hie  w 

ore  know 

led  MV  of  radar  1 .  ■  Mid 

'in"; 

t  beo  r 

v  in  id 

prucc- 

dn its.  Noll,  probleiu-'olving  ■uni  uvliMirai  i a  f,  run ( inn  items  wole 
list'll ,  as  illilsl  i';i  ted  by  the  l  u  o  sample  •<  c  ns  below. 

vSe.  in  pin  :  SO.  On  •■<  aye.  el  no  no  us  bombi  ag  run  using  the  Eagle -Nor  dim 
Sighting  Angle  Computer,  the  bomb  is  dropped  by  the — 

89-A  Eagle-Nonler.  Sighting  Angle.  Computer. 

89-B  operator’s  indicator  computer. 

S9-C  radar  observer’s  toggle  switch. 

89-1)  bombardier’s  toggle  switch. 

89-E  bombsigbt. 

109.  If  it  takes  ‘JO  s'vnnds  for  a  bomb  to  reach  the  ground  from 
an  airplane  traveling  at  a  ground  speed  of  ISO  m.p.h.,  what  is  the 
whole  range? 

.10 9- A  4,020  feet. 

109-13  4,180  feet. 

109-C  5,090  feet. 

109-D  5,280  feet. 

109  -E  0,080  feet. 

lb  ne  e  the  i  piipnu-nt  and  assoeialed  !  raining  course  was  developed 
1  .te  111  ll  ’  rai'ier  progl  ale,  only  one  form  of  !hb  lost,  W  Plg-A,  was 
] > r- -p:: i ed.  'This  hum  was  in.  tailed  ju-4  l »» •  f* •  "e  the  end  of  tho  Avar, 
so  \  i  vy  few  si  a  1 1:  lira!  data  wi  re  a  emmilat  <‘d  for  it.  i 


i\u\  i  '  ll  ion  'i’rolieb  ;icy  'Test 

The  Navigation  Proficiency  Te.- 1  K  mmsim  s  knowledge  of  nonradar 
i::u  boat  mu  i:\foi  m:d  uui  and  teelmique.  dial  are  considered  to  be  essen* 
tml  for  potential  l  el  r  no  ■  rvers.  d'h  '  t •  - 1.  does  not  include  items 
pertinent,  to  i  dnr  opera? ion  or  radar  navigation,  but  is  merely  a 
general  lmvig  it  ion  proficiency  measure. 

'1  ho  Nn\  i;y.t  ion  Proiieiency  Ted  wa  :  iii >1  used  in  the  1G week  radar 
oh. 'Tver  cm  riciilmn,  1  .-n.g  given  as  a  final  e;  animation  at  tho  closo 
of  the  kweek  navigation  pie. ice.  Wlmu  this  phase  was  eliminated 


51  'ibla  U  ,t  was  ilrvrlojuil  by  Cnpt.  Ike  II.  ttnrrluon.  He  v.nB  nssluteil  by  3gt.  Kriwlt. 


88 


1 


uni]  tin*,  curriculum  was  reduced  to  10  weeks,  tlie  test  was  occasionally 
,, j veil  to  bombardiers  entering  radar  observer  training  to  indicate  any 
.]■.«,  i(ie  weaknesses  in  navigation.  Finally,  it,  was  incorporated  into 
the  radar  observer  selection  battery  to  eliminate  bombardiers  with 
insiiflieient  navigation  proficiency,  as  discussed  in  chapter  10. 

Much  of  the  item  material  for  the  Navigation  Proficiency  Tost  was 
obtained  from  tests  developed  by  the  Navigator  Project.  This  ma¬ 
terial  was  supplemented  by  navigation  items  from  the  Aircrew  Evalu¬ 
ation  and  Research  Detachment  No.  1  examinations.  The  informal 
quizzes  used  in  the -1-week  navigation  phase  of  the  1  (5-week  curriculum 
nml  various  navigation  manuals  provided  further  suggestions  for 
items. 

The  two  forms  of  this  test,  P5-A  and  Po-B,  contain  148  and  185 
multiple-choice  items,  respectively,  and  require  185  and  125  minutes 
to  administer.  The  test  is  divided  into  seven  parts,  including  three 
sections  which  involve  primarily  problem-solving  items  and  four 
which  consist  mainly  of  technical  knowledge  items.  The  seven 
parts  as  they  appear  in  form  P5-B  are  described  in  the  following  ' 
paragraphs. 

Part  I  is  a  speeded  section  including  40  E-GB  computer  problems 
with  a  time  limit  of  40  minutes.  The.  items  include  such  problems  as 
determining  true  air  speed,  ground  speed,  wind  force  and  direction, 
track,  and  time  to  destination. 

Part  II  is  a  speeded  section  measuring  skill  in  using  tho  air-plot 
method  to  determine  winds.  It  contains  20  items  with  a  time  limit  of 
35  minutes.  The  items  require  determination  of  either  air  position, 
ground  position,  or  tho  wind  force  and  direction.  Part  of  the  air- 
plot  items  are  solved  on  Mercator  charts  included  in  tho  test  booklets, 
most  of  the  information  is  given  for  tho  solution  and  no  plotting  is 
required.  The  rest  of  the  air-plot  items  require  plotting  on  a  separate 
Mercator  sheet  and  arc  similar  to  tho  air-plot  items  used  in  Final 
Test  I. 

Part  III,  not  speeded,  consists  of  25  technical  information  items 
with  a  time  limit  of  12  minutes.  The  items  measure  knowledge  of 
maps  :md  associated  terminology.  Part  IV,  also  not  speeded,  contains 
1<>  items  with  a  10-minute  time  limit  and  evaluates  understanding  of 
wind  and  its  effect  on  drift.  Tho  items  require  interpretation  of 
information  included  in  vector  diagrams. 

Part  V,  not  speeded,  contains  seven  problems  measuring  knowledge 
of  compass  and  drift-meter  calibration  and  alignment  and  lias  a  time 
hmit  of  8  minutes.  Part  VI,  not.  speeded,  has  22  items,  a  time  limit 
of  10  minutes,  and  measures  knowledge  of  navigation  instruments 
mwh  ns  tho  compass,  altimeter,  drift-meter,  and  air  speed  indicator. 

1  ai't  VII  contains  11  items  to  he  answered  in  10  minutes.  Tho  items 
are  concerned  with  technical  knowledge  and  interpretation  of  tho 
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flux-gate  coinpa  js,  which  is  llu*.  i rn £>< irt ant  u:i:(  of  ! In*  ua.iinit h  i  il »i  1  i- 

zalion  circuit  of  the  air-borne  radar  .set. 

The  revision  of  form  P5  A  of  the  ’ivigaiion  I ’ rot i<  i< •  i n/y  lVst 

which  resulted  in  form  R5 -B  was  iivnli  m-  '"  .  are  when  it  was  decided 

•/ 

to  give  tlic  test  to  boinlmrdiers  fur  the  pm-po-es  of  dingia  is  of  navi¬ 
gation  weaknesses  and  selection  for  radar  ub-ervor  training.  'The 
revision  consisted  of  eliminating  from  the  original  form  all  ’terns 
which  contained  information  not.  taught  in  the  navigation  plumes  of 
bombardier  t  raining. 

The  means  and  standard  deviations  of  selected  part  and  total  scores, 
based  upon  administrations  to  entering  bombardiers,  bombardiers 
who  had  completed  the  4-wcek  navigation  phase  of  radar  observer 
training,  and  previous  navigation  instructors  are,  presented  in  table. 
5.14.  As  would  bo  expected,  the  entering  bombardiers  had  the  lowest 

Tam.’.'  5.14. — Menus  and  standard  deviations  uf  distributions  vj  ruio  score-:  {torn 
navigation  Proficiency  Teat,  P5  form  R,  for  differently  trained  groups* 


Group 

B 

Part  1 

Pari  II 

farts  III-V1I 

Total 

Mean 

si>  ! 

Mean 

SI) 

Mean 

3D 

I  . 

211 

31.. 15 

j 

1.  7.4 

•10. 01 

94.  23 

II. 51 

7  . 

00 

2.5.  Cl 

■Ml 

•1  73 

.'■0.  27 

■PR1 

SO  59 

19.  37 

3 . :... 

32 

32.  22 

4.77  1 

1  14.72 

Z7J 

eo.25 

5.  22  i 

107.  19 

9.09 

1 

1 

1 

•  n roup  1  muslin  nf  bonilordli-ri  compli'tlnR  4  wu‘k:i  of  n  r.  I  !<>o  trnlnltiR,  y  Klrld  class  45-4 It. 

Or..ii|)  l.wif  tv  r  .  I  incl.-y  vi.-M,  2.5  1-Vbmnry  101.5.  Group 

3  roaOsls  o(  previous  n.tvlyillon  Injlructois  o utcrlnu  ru<l..r  ob.crvcr  train. n;;  Langtry  Field,  25  February 
1015. 

scores  in  all  sections  of  the  test.  Tho  scores  of  the  bombardiers  who 
had  completed  the  4  weeks  of  navigation  training  indicate  that  this 
training  was  adequate  to  bring  their  IC— Gi3  performance  almost  to  the 
level  of  well-trained  navigators.  Tho  critical  ratio  of  the  dilTcrencc 
between  tho  means  of  tho  4  week-trained  bombardiers  and  tho  naviga¬ 
tion  instructors  for  part  I,  tho  E-OB  section,  is  0.72.  The  training 
did  not  bring  the  bombardiers  up  to  the  navigation  instructors  in  air- 
plot  pi oficicncy,  the  di (Terence  between  their  means  for  part  II  having 
a  significant  critical  ratio  of  4.57.  A1  o,  the  training  did  not.  appear 
to  improvo  tho  bombardier’s  performance  on  parts  III  to  VII,  tho 
technical  information  sections.  The  difference  between  the  means  of 
tho  entering  bombardiers  and  4-wcck-t rained  bombardiers  for  the 
total  of  the  o  sections  is  insignificant,  having  a  critical  ratio  of  0.02. 

SUMMARY 

Tn  tho  development  of  a  standardized  technique  for  determining 
relative  proficiency  of  students  in  radar  training,  a  battery  of  11 
measures  was  prepared.  I'ivo  of  these  measures  were  printed  pro¬ 
ficiency  tests  and  six  wero  performance  checks.  In  this  chapter  only 
tho  printed  tests  lmvo  been  discussed. 
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[hinted  tests  were  « K- vi*!<>[ <od  for  tile  four  in.-t motional  areas  of 
c  observer  training:  Set  operation,  radar  navigation,  radar 
bombing,  and  radar  intelligence.  Two  types  of  items  characteristic 
of  printed  tests  were  used:  Verbal  questions  measuring  technical 
information,  and  problem-solving  items  duplicating  parts  of  a  stu¬ 
dent’s  actual  job  as  a  radar  observer.  Problem-solving  items  were 
constructed  as  isolated  problems  and  as  series  of  interdependent  prob¬ 
lems,  the  latter  use  simulating  the  sequential  activity  required  in 
actual  radar  operation.  It  was  realized  that  many  of  the  complex 
operations  required  of  the  radar  observer  could  not  bo  effectively 
reduced  to  printed  question  form  but  required  performance  checks 
for  their  evaluation.  In  addition  to  handling  the  equipment  and 
solving  problems  sequentially,  operators  must  interpret  moving  pat¬ 
tern.  on  the  PPI  scope  and  must  recognize  and  adjust  for  peculiar 
symptoms  caused  by  set  malfunctions.  It  is  impossible  to  represent 
moving  patterns  and  malfunctional  symptoms  on  a  printed  page. 

An  outline  of  the  usual  procedure  followed  in  constructing  tests 
and  making  revisions  was  presented.  Material  for  items  was  obtained 
from  existing  informal  classroom  quizzes,  tests  constructed  by  the 
National  Dofcnsc  Research  Committee  and  the  Air-Crew  Evaluation 
and  Research  Detachment  No.  1,  course  outlines,  lecture  notes,  and 
(raining  manuals.  Items  were  reviewed  by  expert  radar  observers  and 
instructors  in  pertinent  coui’ses  before  being  incorporated  into  a  test. 
Problem-solving  items  were  solved  by  panels  of  experts  who  deter¬ 
mined  correct  answers  and  desirable  alternative  choices.  Initial  time 
limits  were  set  on  tho  basis  of  the  time  required  by  experts  to  answer 
all  items  and  were  revised  later  on  the  basis  of  raw  score  distributions 
from  administrations  to  students.  Problem-solving  sections  were 
.speeded  for  the  purpose  of  discriminating  between  students  on  tho 
basis  of  speed  as  well  as  accuracy.  Technical  information  sections 
were  given  adequate  time  limits  to  allow  every  student  to  answer  nil 
problems.  All  tests  were  scored  in  terms  of  the  number  of  correct 
r*-  ponses.  Revisions  were  made  on  the  basis  of  statistical  analysis, 
reliability  studies,  and  examination  of  raw-score  distributions,  and 
on  die  basis  of  systematically  gathered  criticisms  and  changes  in  the 
canicula. 

Tho  principal  diilicultics  in  applying  proficiency  tests  aroze  from 
the  rapid  changes  in  air-borno  radar  equipment  which  resulted  in 
cmr.eular  revisions  and  changing  emphases  on  various  procedures. 
Uio  radar  intelligence  proficiency  tests  were  particularly  nlTa  ted  by 
die  changing  subject  matter  so  that  the  attempt  to  apply  standardized 
measures  to  this  area  of  instruction  was  finally  abandoned. 

Sinee  it  was  not  feasible,  to  describe  all  forms  of  the  test  am  trueted, 
discussion  was  limited  to  typical  tests. 


91 


]'\>r  i"i<  li  t  %  [>0  of  test  ;i  ,s|  nteineiit  of  ( 1 10  gimernl  ! n formation  meas¬ 
ured  \v;is  lii'.  t  presented.  This  \v:is  followed  1  »y  locating  elm  pine:; 
in  tin;  course  of  slndy  where  the  ‘est  is  administered.  A  develop¬ 
mental  history  of  the  measure  was  presented  and  reference  was  made 
to  antecedent,  tests  in  the  same  general  field  of  study.  Following  this 
information  was  a  detailed  description  of  a  typical  form  of  the,  test: 
Number  of  items,  lime  limits,  and  specific  information  measured  by 
tin’  items.  Items  are.  classified  either  as  problem-solving  or  technical 
information.  Tim  description  of  the  type  of  response  required  was 
presented  with  tho  method  of  ascertaining  tolerances.  A  sample 
item  was  presented  with  each  of  those  item  db  cim  mns  to  illustrate 
tho  principles  involved.  For  reference,  all  forms  of  each  test  were 
presented  with  their  respective  codo  numbers.  With  this  list  was 
given  the  reasons  for  the  various  revisions. 

Wherever  possible,  statistical  data  were  presented  for  the  major 
forms  of  each  test.  These  data  included  means,  standard  deviations, 
reliability  coefficients,  and  difficulty  level  and  internal  consistency 
st  atistics. 

Several  recommendations  may  be  made  which  stem  from  the  ex¬ 
perience  of  tho  F.a.dar  Project  in  constructing  standardized  proficiency 
teats.  First,  tho  nature  of  equipment  and  technique  advances  in  air¬ 
borne  radar  requires  that  a  systemized  test  revision  program  be 
adopted.  Already  new  and  improved  equipment  has  made  obsolete 
a  largo  proportion  of  existing  measures.  For  example,  the  original 
estimate  of  necessary  tests  for  existing  equipment  differences  war  16. 
Independent  operating  procedures  and  curriculum  revision  mush¬ 
roomed  test  construction  to  29  forms.  Coordinated  changes  in  pro¬ 
cedure  and  curriculum  might  lmvo  reduced  this  number. 

Second,  the  success  of  a  proficiency  program  depends  heavily  upon 
standardized  tcr.t  administration.  It  is  recommended  that  examiner 
ho  ml:  he  trained  and  e.  tublishcd  wherever  such  a  program  is  initiated. 
Tim  hoard  will  reduce  to  a  small  group  the  individuals  who  must  bo 
convinced  of  the  value  of  standardization. 

When  icing  groups  of  interdependent  problem-solving  items,  care 
must  bo  taken  to  guard  against  unreliability.  Tho  interdependence  of 
il<  ms  may  in  effect  shorten  tho  length  of  a  test  by  automatically 
penalizing  .students  for  a  whole  scries  when  they  miss  tho  first  item. 
The  organizational  and  sequential  aspect  can  be  retained  to  somo  ex¬ 
tent,  by  periodically  breaking  tho  sequence  at  tho  end  of  a  typical 
cries  of  items  and  having  the  students  start  another  group. 

Fxpovionco  with  wind  problems  and  problems  answered  in  terms 
of  latitude  and  longitude  suggests  that  velocity  and  force  or  latitude 
and  longitude  ho  presented  ns  separato  items.  Another  method  of 
lengthening  n  test  without  appreciably  increasing  time  limits  consists 
of  making  separate  items  out  of  intermediate  steps  in  tho  solution  of 
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;1  complex  problem.  "Wherever  possible,  maps,  diagrams,  pictures,  or 
an v  oilier  device  (hat.  adds  reality  to  a  problem  should  be  used. 

A  clear  understanding  of  tho  relative  efficiency  of  printed  tests 
iml  performance  checks  should  be  kept  in  mind  when  selecting  tho 
testing  device  for  a  given  area  of  study.  It  is  recommended  that 
[Direct  answers  for  problem-solving  items  be  determined  by  a  panel 
of  authorities  rather  than  depending  upon  average  results  of  trial 
administrations.  In  determining  differences  between  alternative 
choices  of  problem-solving  items,  it  is  recommended  that  rule,  of- 
tlmmb  differences  be  avoided  and  that  the  proper  deviations  bo  de¬ 
termined  empirically. 

Tho  time  required  for  instructing  students  in  any  single  topic 
should  not  he  considered  as  the  index  of  the  importance  of  that  topic. 
Relative  importance  of  subject  matter  should  be  determined  by  train¬ 
ing  authorities. 

Experience  with  problem-solving  items  suggests  that  speed  as  well 
as  accuracy  be  measured.  For  determination  of  reliability  of  these 
speeded  forms,  alternate  forms  must  bo  prepared. 


CHAPTER  SIX _ : _ 

Standardized  Performance  Checks' 


INTRODUCTION 

The  necessity  of  supplementing  printed  proficiency  tests  wilh  meas¬ 
ures  of  actual  performance  with  radar  equipment  soon  became  clear. 
In  determining  the  student’s  course  grade,  radar  school  authorities 
weighted  heavily  student  performance  on  ground  trainers  and  aerial 
lnirmons.  If  course  grades  were  to  be  mado  more  reliable,  standard¬ 
ized  measures  of  performance  had  to  be  provided. 

Considerable  contrast  is  evident  in  comparing  tho  classroom  situa¬ 
tion  in  which  printed  tests  are  given  with  the  actual  job  conditions 
of  the  radar  observer.  The  classroom  is  usually  comfortable,  with 
plenty  of  space  to  work,  and  tho  student  has  no  responsibility  othor 
than  attempting  to  do  well  on  the  test.  The  working  situation  in  tho 
air  is  vastly  different.  •  Tho  student  is  usually  hampered  by  cumber¬ 
some.  clothing;  he  must  wear  an  oxygen  mask;  his  working  space  is 
cramped  and  inadequate;  and  he  is  subject  to  all  tho  usual  distractions 
and  anxieties  of  flying.  Moreover,  ho  has  the  responsibility  of  direct¬ 
ing  the.  aircraft  and  cooperating  with  tho  other  crow  members. 

Performance  in  the  air  and  on  tho  ground  trainer  differs  further 
from  a  written  test  situation  in  that  it  is  continuous  and  paced. 
Th:  •ardloss  of  what  the  student  does,  tho  aircraft  keeps  moving,  re¬ 
quiring  new  computations  and  procedures.  lie  cannot,  as  he  may 
on  a  printed  test,  answer  the  easy  problems  first  and  delay  tho  solu¬ 
tions  of  difficult  problems.  Since  the  various  aspects  of  radar  per¬ 
formance  in  the  air  are  interdependent,  failure  to  perforin  adequately 
on  a  difficult  aspect  will  likely  detract  from  performance  on  other 
aspects  which  may  be  easier. 

Stand  ardi/.cd  performance  checks  were  indicated  for  three  aspects 
of  radar  observer  training:  Performance  in  tho  air,  at  the  supersonic 
trainer,  and  at  the  bench  set  trainer.  It  was  decided  in  consultation 
with  training  authorities  that  aerial  performance  would  be  checked 
at  two  levels.  The  first  aerial  check  would  be  an  intermediate  check 
to  he  administered  on  a  mission  approximately  half  way  through  the 
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student's  aerial  (raining.  '1  ho  second  \v< >11 1<  1  be  a  final  check  given 
just.  prior  to  completion  of  flic  course.  Supersonic  trainer  perforrn- 
anee  waste  be  evaluated  at  tl  "re  points.  An  intcrinediiite  navigation 
cheek  was  to  1>'  given  after  instruction  in  Italic  navigational  pro- 
cedure  ;  an  intermediate  bombing  cheek  a  fter  bombing  l(  cluiiques  were 
taught  ;  and  a  final  check  near  the  completion  of  s.upci  sonic  training. 
Performance  on  the  bench  set  trainer  was  to  he  covered  by  one  final 
check  to  be  given  relatively  early  in  the  course  after  the  elements  of 
set  operation  had  been  taught. 

In  this  chapter  an  illustrative  form  of  each  of  these  three  typos  of 
cheeks  will  he  described  and  available  statistical  data  will  he  pre¬ 
sented.  In  addition,  a  brief  discussion  will  present  some  problems 
encountered  in  constructing  performance  cheeks  and  in  training  ex¬ 
aminers  to  administer  them.  Consideration  of  systematic  aspects  of 
the  measurement  of  performance  will  be  deferred  until  chapter  7. 
Tlio  present  chapter  is  intended  only  as  a  history  of  the  development 
of  the  performance  cheeks  in  n  specific  training  situation. 


GENERAL  PROCEDURE  USED  IN  DEVELOPING  PERFORM- 

ANCE  CHECKS 


Before  outlining  the  procedure  followed  in  constructing  the  specific 
performance  measures  developed  by  Psychological  Research  Project 
(Radar),  a  general  characterization  of  “performance  checks”  is  in 
order.  A  performance  cheek  is  usually  administered  to  one  individ¬ 
ual  at  a  time  by  a  trained  examiner.  The  student  is  required  to 
perform  n  standardized  series  of  tasks  which  is  the  same  or  equivalent 
for  all  students.  The  conditions  under  which  each  task  is  to  be 
accomplished  are  also  standardized.  For  each  task,  standards  of 
success  and  other  instructions  for  administration  are  prescribed. 

The  first  stop  in  constructing  each  check  for  the  radar  observer 
program  was  the  selection  of  the  critical  aspects  of  performance.  The 
personnel  charged  with  the  development  of  a  particular  check  began 
by  systematically  observing  the  behavior  to  be  measured.  For  ex¬ 
ample,  before  starting  work  cm  the  aerial  performance  checks,  project 
members  accompanied  student  and  graduate  radar  observers  on  numer¬ 
ous  llights.  In  addition  to  observation,  the  task  was  also  studied  by 
participation;  those  responsible  for  check  construction  learned  and 
took  part  in  the  performance  themselves.  To  this  end,  many  members 
of  the  project  took  the  entire  radar  training  course,  while  others  con¬ 
centrated  on  only  those  phases  of  the  course  with  which  they  were 
mainly  concerned.  Frequent  conferences  were  held  with  instructor 
consultants  and  department  heads.  Their  personal  experience  as 
radar  observers  us  well  as  their  familiarity  with  curriculum  and  in¬ 
struction  were  utilized  to  great  advautugo. 
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Once  the  items  of  lu  liii  vior  to  bl*  checked  u  t  re  r-e 'ir! <  d.  it  u  Us  nc<  OH- 
sny  to  decide  on  the  method  of  measurement  for  cat  h  item.  I low- 
t. se r,  Hiidi  decisions  wert1  so  dependent  on  tin*  task  to  lie  cheeked  that 
(heir  discussion  will  be  postponed  until  the  specific  checks  are  con¬ 
sidered.  After  the  methods  of  measurement  were  sel  cted,  the  check 
items  and  standards  of  success  were  stated  in  preliminary  form  and 
trial  administrations  were  begun.  On  the  basis  of  these  trial  admin¬ 
istrations,  revisions  were  made  in  both  the  choice  of  items  and  methods 
of  measurement. 


ARRANGEMENT  AND  FORMAT 

An  effori,  was  made  to  standardize  the  organization  and  appear¬ 
ance  of  all  performance  checks  developed  by  the.  radar  project.  The 
items  of  performance  to  be  checked  were  preceded  by  a  uniform  in¬ 
troductory  section.  This  section  contained  notes  on  standardized 
performance,  checking,  a  statement  of  required  conditions  for  the. 
check,  and  directions  for  administration  and  for  student  briefing. 
The  notes  on  standardized  performance  checking  stated  briefly  cer¬ 
tain  principles  which  were  expanded  during  examiner  training  lec¬ 
tures.  'Hie  section  on  standard  conditions  enumerated  the  training 
prerequisites  which  the  student  must  have  completed  before  being 
given  the  check;  it  called  attention  to  the  requirement  that  only  trained 
examiners  should  administer  the  check;  it  listed  the  equipment  needed 
by  both  student  and  examiner;  and  it  described  the  conditions  under 
which  the  check  was  to  he  given.  For  each  of  the  aerial  and  super¬ 
sonic  cheeks,  standard  missions  were  prescribed.  For  the  bench  check, 
the  positions  were  listed  in  which  the  controls  of  the  radar  set  were  to 
be  placed  before  starting  the  check. 

The  prescribed  procedures  for  administration  included  directions  to 
the  examiner  for  using  the  special  scoring  sheet,  acquainted  him  with 
his  special  duties  in  administering  the  particular  check,  and  cautioned 
him  to  brief  the  student  carefully.  The  student  briefing  section,  to 
be  read  by  the  student.,  defined  the  ta>k  for  the  student  and  stated  the 
special  requirements  to  he  fulfilled. 

The  introductory  section  was  followed  by  the  items  of  performance 
to  ho  checked.  Each  was  accompanied  by  standards  of  success  and 
methods  of  evaluation.  The  items  appeared  on  cut-hack  pages  which 
^eve  narrower  than  the  other  pages  in  the  check  booklet.  A  single 
answer  sheet  for  all  the  items  followed  the  item  pages  and  was  of 
normal  width.  The  right-hand  side  of  the  answer  sheet  contained 
spares  in  which  a  check  mark  (V)  or  a  zero  (0)  wore  to  he  entered,  de- 
pending  upon  whether  a  student  did  or  did  not  satisfy  the  standards 
of  success  for  the  particular  item.  Because  of  the  narrow  item  pages, 
both  the  items  and  the  check  spaces  were  visible  at  the  same  time, 
bliis  made  it  unnecessary  to  turn  pages  to  check  an  item.  Such  an 
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:m angrment  was  of  advanl age  also  in  that  only  (In:  ;  oring  sheet  was 
expendable.  Two  different  arrangements  involving  cut  back  pages 
were  employed.  On  some  performance  cheeks,  the  item  pages  de¬ 
creased  in  width  on  succeeding  pages  so  that  there  was  provision  for 
several  columns  of  check  spaces  on  the  scoring  sheet.  On  other  per¬ 
formance  checks,  the  item  pages  v.erc  of  uniform  width  and  required 
only  one  column  of  check  spaces.  The  latter  arrangement  was  more 
compact  and  easier  to  reproduce  but  was  limited  to  performance  checks 
in  which  the  number  of  items  did  not  exceed  the  number  which  could 
he  cheeked  in  one  column. 

Several  performance  cheeks  included  a  further  section  which  con¬ 
tained  supplementary  instructions  to  the  examiner.  The  instructions 
were  numbered  to  correspond  with  individual  check  items  and  elabo¬ 
rated  the  more  general  statement,  of  methods  of  checking.  This  sec¬ 
tion  saved  valuable  space  on  the  item  pages  and  avoided  encumbering 
them  with  material  that  was  useful  but  not  always  essential  in  checking 
the  items. 

EXAMINER  TRAINING 

The  effectiveness  of  a  performance  check  depends  heavily  upon 
proper  administration.  Recognizing  this,  the  project  recommended 
th<;  formation  of  examiner  boards  to  bo  responsible  for  all  proficiency 
measurement.  It  was  urged  that  competent  radar  observers,  inter¬ 
ested  in  proficiency  measurement,  bo  assigned  as  members  of  such 
boards. 

The  training  of  examiners  was  initiated  with  a  lecture  covering 
relative  grading,  the  importance  of  standardization,  the  rationale  of 
performance  cheeking,  and  the  distinctions  between  testing  and  teach¬ 
ing.  Specific  cheeks  wero  discussed  in  detail.  Each  item  was  read, 
explained  and  illustrated,  and  the  directions  for  administration  wero 
reviewed.  Examiners  wero  warned  against  common  errors  in  ad¬ 
ministering  each  check.  This  lecture-discussion  was  followed  by  trial 
admini:  [rations  of  each  check  by  pairs  of  examiners,  who  alternated 
as  subject  and  cheek  administrator.  Each  trial  check  was  carefully 
supervised  by  project  personnel.  For  practical  reasons,  trial  adminis¬ 
trations  of  the  aerial  check  were  given  on  tho  supersonic  trainer. 

Since  it  was  impossible  for  project  members  to  supervise  all  subse¬ 
quent  routine  cheek  administrations,  certain  examiners  were  given 
supervisory  responsibility.  It  was  their  duty  to  see  that  conscientious, 
standardized  checking  was  carried  out.  In  addition,  however,  project 
personnel  miulo  frequent  observations  of  check  administration  and 
reported  discrepancies  and  difficulties  to  tho  chief  examiner. 

CHECK  REVISIONS 

Revision  of  the  battery  of  six  standardized  performance  checks  was 
to  be  expected  in  the  new,  expanding  radar  training  program.  An 


98 


evident  rnson  for  now  forms  of  the  chocks  was  tli:it  tho  typos  of  radar 
(*![ti i pinout  varied  from  school  to  school.  Langley  Field,  where  tho 
f:i>t  sot  of  cheeks  was  developed,  taught  the  AN/APS-15  and  tho 
AN/APS  -lf)A  sets.  Boca  Baton  Army  Air  Field,  Victorville  Army 
Air  Field,  Williams  Field,  and  Ynmn  Army  Air  Field  all  used  the 
AN/APQ-13  radar  set.  AN/APQ-7,  a  newer  set  and  less  important 
so  far  as  student  (low  was  concerned,  was  tho  subject  of  a  separate 
course,  centered  first  at  Boca  Baton  and  later  at  Williams  Field. 

The  necessity  for  check  revision  was  not  limited,  however,  to  deal¬ 
ing  with  equipment  differences.  Even  when  two  schools  used  iden¬ 
tical  radar  sols,  there  was  no  assurance,  that  the  procedures  taught  for 
tuning  the  set  or  for  navigating  and  bombing  with  tho  set  would  not 
differ  from  school  to  school.  For  example,  although  tho  schools  at 
Boca  Baton  and  Victorville  both  used  the  AN/APQ-13  equipment, 
tho  method  by  which  the  student  was  taught  to  computo  a  wind  from 
his  radar  data  differed  markedly  between  the  two 'schools.  To  under¬ 
stand  this,  it  should  be  remembered  that  radar  observer  training  had 
developed  without  any  centralized  instructional  authority.  Difficul¬ 
ties  in  developing  standardized  performance  checks  under  such  cir¬ 
cumstances  were  unavoidable. 

Not  only  did  the  curriculum  vary  between  schools,  but  in  any  one 
school  it  was  subject  to  frequent  change.  This  was  a  consequence  of 
the  fact  that  radar  training  was  new  and  better  ways  of  doing  things 
wero  constantly  being  discovered.  Fortunately,  such  curriculum 
changes  were  not  usually  radical,  and  revised  forms  of  the  checks  could 
bo  produced  by  changes  in  several  items.  Occasionally,  however,  a 
curriculum  change  of  such  magnitude  occurred  that  the  use  of  radar 
project  performance  checks  had  to  bo  abandoned  temporarily. 

Equipment  and  curriculum  changes  wero  of  courso  not  the  only 
forces  motivating  check  revisions.  Experience  with  the  initial  forms 
of  the  checks  sometimes  showed  that  the  selection  of  critical  items 
could  be  improved  or  that  the  method  of  measurement  for  items  re¬ 
tained  could  he  revised  for  more  accurate  evaluation.  Also,  item 
analysis  occasionally  demonstrated  that  the  tolerances  set  for  pre¬ 
cision  items  was  too  small  or  too  large,  licvisions  attempted  to  in¬ 
corporate  all  promising  improvements. 


STANDARDIZED  BENCH  SET  PERFORMANCE  CHECKS 


The  Bench  Set  and  tlic  Student’s  Task 
The  apparatus  upon  which  the  standardized  bench  set  check  was 
administered  was  simply  the  radar  set  installed  in  a  classroom,  rather 
then  in  tho  airplane.1  The  parts  of  tho  equipment  with  which  tho 
check  was  concerned  wero  tho  various  controls  involved  in  tuning 

*  For  <i  description  of  tho  Get,  bc«  chapter  4.  ' 
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and  calibrating  the  .set,  the  PPI  scope,  the  “A’’  scope  (except  on  the 
AN/APQ-13  set),  tho  computer  box,  and  various  voltage  controls. 

The  student's  task  on  the  bench  set  was  to  make  the  preoperational 
( heck,  start  and  tunc  tho  set,  calibrate  it  correctly,  and  turn  it  oif. 
The  preoperational  check  included  the  adjustment,  of  approximately 
lit)  controls.  Starting  the  sot  included  turning  on  the  power,  bright¬ 
ening,  foeusing,  centering,  and  in  other  ways  adjusting  the.  systems 
which  presented  tho  ground  returns  on  the  scopes.  Tuning  consisted 
in  picking  up  ground  returns  and  adjusting  the  transmission,  re¬ 
ceiving,  and  presentation  systems  for  maximum  definition  on  the 
scopes.  Turning  the  set  olF  included  the  adjustment  of  several  con- 
t  mis  t  o  avoid  damage  to  the  equipment  when  the  power  supply  was  cut. 

The  objectives  of  bench  set  instruction  were  often  a  source  of  dis¬ 
agreement  among  instructional  personnel.  Some  were  convinced  that 
(In',  student  should  merely  learn  “procedure”  on  the  bench  set:  the 
controls  to  adjust,  tho  direction  and  amount  of  adjustment,  and  the 
sequence  in  which  the  adjustments  should  be  made  in  order  to  produce 
tho  desired  results  on  the  scope.  Others  were  convinced  that  the 
mastery  of  procedure  was  relatively  unimportant  and  that  the  stu¬ 
dent’s  proficiency  should  be  evaluated  in  terms  of  the  end  result,  i.  e., 
tlu>  quality  of  the  scope  picture  resulting  from  his  precision  in  tuning. 

When  the  fu-st  forms  of  the  bench  set  performance  check  were 
being  constructed,  it  was  decided  that  the  emphasis  of  bench  set  in¬ 
struct  ion  should  be  on  tho  learning  of  a  standard  operating  procedure. 
The  first  bench  check  was  therefore  built  to  measure  the  procedure 
typo  of  proficiency.  For  examplo,  tho  item  which  refers  to  centering 
the  trnco  on  tho  scopo  was: 

Center  Sicccp  horizontally  and  vertically  to  reduce  size  of  hole  in  renter  of 
scope. 

No  attempt  was  made  to  evaluato  tho  quality  or  precision  of  tho 
student's  adjustment.  It  was  considered  sufficient  that  he  knew  which 
controls  to  manipulate  to  work  toward  tho  desired  resr.lt.  A  further 
condition  for  receiving  credit  on  the  above  item  was  that  the  student 
made  the  adjustment  in  proscribe  l  sequence  with  other  items  which 
preceded  and  followed  it. 

After  several  months,  the  desire  of  training  authorities  to  measure 
precision  as  well  as  procedure  became  strong  enough  to  incorporate 
into  the  bench  check  somo  items  which  considered  the  quality  of  tho 
student's  adjustment.  For  example,  the  item  on  centering  became: 

Center  Sweep  horizontally  nnd  vertically  In  exact  center  of  scope.  (If  sweep 
Is  not  exactly  centered,  zero  (0)  tills  Item.) 

Tho  student’s  knowledge  of  standard  operating  procedure  still  ac¬ 
counted  for  tl  10  majority  of  items  in  this  check,  but,  wherever  possible, 
items  were  stated  with  accuracy  requirements.  It  is  probable  that 
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i|ii‘  early  pic ference  for  procedure  items  was  associated  with  the  use 
of  the  older  form  of  the  cheek  as  a  teaching  aid.  The  check,  however, 
apparently  improved  and  standardized  instruction  to  a  point  where 
most  students  could  perform  the  standard  operation  procedure  with 
almost,  no  error.  The.  check  had  become  “to  easy,”  and  the  desire 
to  increase  its  difiiculty  partly  accounted  for  the  later  decision  to 
evaluate  precision. 

The  bench  set  check  was  administered  during  the  last  period  allotted 
to  bench  set  instruction  when  the  student  had  completed  training  in 
the  various  elements  if  the  bench  set  task.  Students  could  be  reverted 
to  a  lower  class  if  deficient  in  set  operation,  thus  preventing  the  loss 
of  valuable  aerial  instruction  time. 

Bench  Check  Construction 

The  selection  of  items  for  the  bench  set  check  was  relatively  simple. 
There  was  no  necessity  for  sampling  behavior;  all  the  stops  in  the  task 
could  be  included.  Towards  the  end  of  the  war  the  construction  of 
the  initial  bench  set  check  at  a  school  consisted  primarily  in  stating  the 
standard  operating  procedure  in  check  form.  Of  course,  certain 
changes  in  phrasing  were  made,  and  standards  of  success  for  each  item 
were  stated.  At  the  beginning  of  the  training  program,  however, 
check  construction  was  handicapped  by  a  lack  of  uniform  instruc¬ 
tional  practice  even  within  a  school.  For  example,  at  the  school  where 
the  first  bench  set  check  was  constructed,  no  standard  operating  pro¬ 
cedure  existed  and  the  procedure  a  student  learned  reflected  the  prac¬ 
tice  of  the  instructor  to  whom  lie  was  assigned.  The  efforts  to  estab¬ 
lish  a  standard  measure  of  proficiency  speeded  the  formulation  and 
publication  of  a  standard  operating  procedure.  Since  this  procedure 
was  to  be  integrated  into  an  aerial  check,  standardization  between 
flight  line  and  ground  school  instruction  was  also  promoted. 

The  Typical  IT  rich  Set  Check 

(ho  typical  bench  set  performance  check8  contains  the  conven¬ 
tional  introductory  material  as  outlined  above.  The  most  important 
section  of  the  introductory  material  directs  the  examiner  to  set  about 
BO  controls  in  designated  positions.  Most  of  these  settings  were  de¬ 
signed  to  put  the  cont rols  out  of  adjustment  so  that  the  student  would 
he  required  to  readjust  them.  The  controls  were  to  be  set  in  the  same 
initial  positions  for  every  student. 

The  number  of  items  in  the  various  forms  of  the  bench  set  check 
varied  from  79  to  BIG.  As  indicated  above,  they  included  all  items 
necessary  to  accomplish  the  tuning  and  calibrating  tasks.  The  student 
earned  a  check  on  an  item  if  he  performed  the  task  as  stated  and  if  he 
performed  it  at  the  proper  time.  Sequence,  therefore,  was  very  im¬ 
portant.  Most  of  the  steps  could  be  performed  correctly  only  between 

1  See  nppcmMx  B. 
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two  other  specific  steps.  Small  groups  of  items  were  sometimes 
bracketed  which  indicated  that  the  items  within  the  brackets  could  be 
performed  in  any  sequence. 

Types  of  Bench  Check  Items 

As  mentioned  above,  procedure  items  constituted  the  bulk  of  the 
bench  check.  An  example  of  a  pure  procedure  item  follows: 

Rt'rvivcr  Gain:  Counter-clockwise. 

No  attempt  was  made  to  give  more  credit  for  important  procedure 
items  than  for  minor  ones.  In  later  forms  of  the  check,  as  already 
noted,  examiners  were  required  to  judge  the  precision  of  adjustments. 
For  example: 

Turn  up  PPI  llrilliancc  until  trace  Is  barely  visible.  (If  trace  is  Invisible  or 
too  bright,  zero  (0)  this  item.) 

In  addition  to  items  of  actual  performance,  most  of  the  bench  checks 
contained  a  few  items  that  were  administered  as  oral  questions  during 
the  course  of  the  check.  For  example: 

Instructor  asks:  “In  the  air  what  must  you  do  immediately  after  turning  the 
power  on?- 

Answer:  “You  must  select  Inverter  No.  1  or  No.  2  by  means  of  the  Inverter 
selector  switch." 

One  reason  for  such  questions  was  that  certain  essential  steps  in 
aerial  procedure  could  not  be  checked  on  the  bench  set  because  the  ap¬ 
propriate  equipment  was  not  at  hand.  For  instance,  in  reference  to 
the  quoted  item,  there  were  no  inverters  on  the  bench  set.  Other 
questions  were  directed  to  asking  the  student  what  ho  would  do  if 
certain  expected  results  did  not  occur.  The  student  was  not  required 
to  answer  in  the  exact  words  given  for  the  item,  but  lie  had  to  include 
all  essential  elements  of  the  answer  in  his  reply.  Since  the  early  fc  'ms 
of  the  bench  set  check  wero  used  for  instruction,  tho  questions  ;  nd 
answers  were  just  another  item  of  procedure,  which  the  student  could 
memorize. 

Bench  Check  Administration 

In  administering  the  check,  tho  examiner  was  instructed  to  say 
nothing  to  tho  student  except  that  which  appeared  in  quotation  marks 
in  tho  check.  Except  for  the  items  in  question  form,  tho  examiner 
gave  only  general  infrequent  directions  such  as  “Go  through  tho  pro- 
operational  check.”  The  student  was  briefed  to  perform  all  tho  steps 
called  for  by  tho  examiner’s  directions  and,  as  an  aid  to  checking,  to 
verbalize  nil  steps  as  ho  did  them.  When  tho  student  performed  an 
item  correctly  and  in  tho  correct  sequence,  the  examiner  entered  a 
check  mark  in  tho  appropriate  space  on  the  scoring  sheet.  If  tho  item 
was  performed  incorrectly,  omitted,  or  not  performed  in  the  correct 
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.■.pquonce,  u  zero  was  entered.  When  the.  student  performed  tho  item 
ii;ri>rrec:tly  or  omitted  it,  the  examiner  corrected  him  at  once  and  told 
him  the  adjustment  to  make.  The  administration  of  the  check  re¬ 
quired  about  one-half  hour. 

In  .-pi to  of  the  apparent  simplicity  of  the  checking  procedure,  there 
was  some,  difiiculty  in  securing  standard  administration.  Differences 
in  the  functioning  of  equipment  were  common.  Some  sets  on  which 
checks  find  to  he  given  would  not  pick  up  targets.  This  reduced  cer¬ 
tain  steps  in  the  timing  to  artificial  procedure  instead  of  actual  work 
with  a  radar  return.  Meter  readings  on  the  bench  set  were  frequently 
outside  the  acceptable  range,  and  the  student  could  not  adjust  them 
because  of  the  nature  of  the  power  supply  on  the  ground. 

Kxaininer  differences  were  also  encountered.  Halo  effect,  accentu¬ 
ated  by  some  students’  apparent  confidence  and  ability,  led  examiners 
to  give  credit  to  supposedly  good  students  whose  errors  wero  ex¬ 
plained  away  as  only  oversights.  Sometimes  less  conscientious  ex¬ 
aminers  who  disagreed  with  the  standard  operating  procedure  would 
give  the  student  credit  for  following  a  procedure  which  suited  tho 
examiner  rather  than  the  chock.  Some  examiners  were  noticed  giving 
unintentional  hints  or  leading  questions  in  a  well-meaning  effort  to 
“get  the  best  out  of  the  student.”  Operating  in  the  other  direction,  a 
common  mistake  was  failure  to  stop  the  student  at  once  for  an  error 
or  omission  and  thus  permit  his  errors  to  multiply. 

Bench  Check  Revisions 

It  has  been  noted  that  new  forms  of  tho  bench  check  wero  needed 
because  of  equipment  differences,  changes  in  curriculum  and  operating 
procedure,  aud  the  increased  emphasis  on  precision.  In  all,  li  forms 
of  the  Final  Bench  Set  Check  were  published.  Three  of  theso  (PGa- 
A,  POa-B,  and  PGa-C)  applied  to  the  \N/APS-15  equipment  at 
Langley  Field,  two  (PGb-A  and  PGb-B)  applied  to  t he  AN/APS- 
1"»A  at  Langley  Field,  two  (PGc-A  and  PGc-B)  applied  to  tho 
AX/APQ-13  as  taught  at  Boca  Raton,  two  (PGc-C  and  V  PGc-B) 
applied  to  the  AN/APQ-13  at  Victorville,  one  (W  PGc-A)  applied  to 
the  AN/APQ-13  at  Williams  Field,  and  one  (W  PGd-B)  applied  to 
tho  AN/APQ-7  at  Williams  Field.4 

Bench  Check  Statistical  Findings 

Figure  G.l  presents  a  typical  distribution  of  scores  on  the  bench 
check.  The  total  possible  score  on  this  form  was  13G.  The  mean  of 


*  Don  :l.is  W.  Tlrny,  Si;t.  Gerald  S.  Hlum,  S/Sgt.  lUelinrd  T.  Mitchell,  S^t.  Albert  fl. 
llT-torf.  uinl  Si;t.  llymnn  1 1  !<-r  collaborated  In  developing  tlio  first  lb-ncli  Set  Check 

(Pr.n-A).  nicht  of  tlie  11  checks  were  constructed  by  Cpl.  llrny  and  Hgt.  1  tin  in.  Tho  r»- 
tralnlng  two  wire  developed  by  Sgt.  Stanly  mid  Sgt.  Nathaniel  L.  Gage. 


703327—47 


8 


103 


RAW  SCORE 


N  - 

428 

Skew  ■  “ 

•  2.00 

Mean 

■  122.74 

S03K  - 

.45 

SD 

•  9.32 

gr„k- 

4.44 

Fir.uiu:  O.L — Frequency  distribution  of  raw  scores  for  a  standardized  final  bench 
check  (PGa  -B) — administered  at  Langley  Field. 


122.7  shows  that  the  check  was  easy  for  most  students.  Note  tho  re¬ 
stricted  range  and  tho  significant  skewness  of  tho  distribution.® 

Since  the  reliability  of  bench  check  scores  depended  in  largo  mea¬ 
sure  ou  standardized  administration,  there  was  a  desire  to  examine 
chock  data  statistically  in  an  attempt  to  determine  whether  such  stand¬ 
ardiz'd  ion  had  been  achieved.  It  was  anticipated  that  the  technique 
of  analysis  of  variance  would  bo  employed  to  test  the  null  hypothesis 
that  there  was  no  significant  difference  between  mean  check  scores 
given  by  different  examiners.  For  this  purpose,  data  on  accredited 
examiners  who  had  administered  six  or  more  bench  checks  were  ac¬ 
cumulated  for  hath  Langley  Field  and  Boca  Raton.  These  data  are 
presented  in  tables  G.l  and  6.2. 

A  chi-sqnure  test  for  homogeneity  of  variance  revealed  that  tho 
analysis  of  variance  technique,  could  not  be  used  on  either  set  of  data. 

•Tor  tlila  and  Htibseiiurnt  distributions  skewness  wna  computed  from  the  following 
formula  : 

POT-^-Pi»  where  PM  la  the  00th  centlle,  Pi0  la  the  10th  centlle  nnd  Is  the  50th 

■centlle.  Standard  deviation  of  the  measure  of  skewness  was  computed  as  follows: 

o  r.  i  q  a 

where  D=iP„~Pla. 

Kelley,  T.  ...  Statistical  method,  1023,  p.  7T. 
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T.uii.k  0.1.  —l'i-rn miner  mean*  and  standard  drriat  ions  for  ii  standardized  bench 
set  trainer  performance  eked:  (P6a  11)  administered  at  Lanylcy  Field 
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4.21 
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53 

121.  26 

10.  41 

•i.S 
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7.  23 

1. . . 

42 

120.  00 

11.55 
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13 

123.  li'J 

22.  11 

M . 

22 

113.77 

7.  IS 

•13 

123.  2ll 

10.53 

N . 

10 

lit.  10 

13.04 

13 

123. 

9.  74 

" . 

3t! 

123.  H 

10.  17 

Total . 

303 

122.49 

9.80 

i  The  highest  possible  raw  score  on  this  check  was  136. 


T.uii.k  (5.2. — Examiner  means  and  standard  deviations  for  a  standardized  bench- 
set  trainer  performance  check  [PCc-B)  administered  at  Boca  Raton 
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raw 
scoro  1 

SD 

Eiamlncr 

Number  of 
checks  ad¬ 
ministered 

Mean 
raw 
score  i 

SD 

7 

78.  -13 

0.49 

K . 

8 

73. 25 

1.71 

\\  . 

11 

73.00 

.74 

L . 

■ppS 

” ~ 

7Z43 

6.  12 

c  . 

17 

77.00 

2. 85 

M . 

4.  63 

I) . 

13 

75.  54 

2.44 

N . 

rj 

71.85 

0.28 

K  . 

•  2 

74.92 

2.  78 

O . 

71.  14 

4.  lb 

y  . 

9 

74.33 

2.45 

P._ . 

70.  29 

6.25 

0  . . 

15 

74.  27 

3.70 

Q . 

Stari 

70.29 

4.  02 

n . 
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10 

73  40 

b  41 

11 

73.  27 

2L70 

Total . 

201 

73.  C3 

4.02 

i  The  highest  possible  raw  score  on  this  performance  check  was  79. 


The  examiner  means  in  both  tables  show  considerable  variation, 
probably  more  than  would  bo  expected  by  chance.  The  variations 
among  the  examiner  sigmas  is  even  more  striking.  In  fact,  the  tables 
show  not  only  that  examiners  varied  in  the  average  score  given,  but 
also  that  some  examiners  gave  scores  in  a  much  more  restricted  range 
than  others.  An  inspection  of  the  tables,  particularly  table  G.2,  sug¬ 
gests  a  negative  correlation  between  means  and  sigmas.  This  may  be 
due,  at  least  in  part,  to  the  closeness  of  most  of  the  scores  to  the  highest 
possible  scoro  on  the  check,  leaving  little  room  for  upward  variation. 

In  attempting  to  interpret  the  examiner  data,  tho  assumption  is 
made  that  students  checked  by  an  examiner  are  a  random  sample  of 
the  population  of  students.  While  it  is  not  known  that  any  factors 
produced  a  biased  sample  of  students  for  any  examiner,  neither  can 
it  ho  stated  that  any  precautions  against  a  bias  wero  taken.  With 
this  qualification,  it  seems  likely  that  the  examiner  means  and  sigmas 
(1o  not  have  the  consistency  which  one  would  desire  from  standardized 
administration. 

As  pointed  out  earlier,  high  scores  on  the  bench  checks  led  to  a 
greater  emphasis  on  the  inclusion  of  precision  items.  Another  remedy 
proposed,  but  not  actually  applied,  was  that  the  amount  of  time  tho 
student  required  to  go  through  the  check  be  considered  in  his  grade, 
lo  examine  this  possibility,  time  required  to  complete  the  cheek  was 
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recorded  on  the  scoring  sheets  for  several  cI.isk-s  at  Victorville  Army 
Air  Field.  A  coefficient  of  eorrehilion  computed  Let  ween  these  time 
scores  and  the  raw  error  scores  on  a  sample  of  "09  e:w.  was.  0.15 
This  correlation  is  low  enough  to  suggest,  that  11  combination  of  cheek 
and  time  scores  would  produce  a  more  discriminating  total  grade. 

There  wero  difficulties,  however,  in  timing  the  bench  check.  Equip- 
ment  difTeronces  influenced  the  speed  with  which  the  student  could 
perform  the  required  steps.  For  instance,  if  it  were  more,  difficult 
to  pick  up  a  target  on  one  set  than  on  another,  equally  proficient 
students  might  earn  different  time  scores.  Also,  the  examiner  might 
influence  student  speed  sineo  he  was  required  io  stop  the.  .student  and 
correct  him.  Before  time  scores  can  be  used  to  grade  students,  both 
equipment  and  examiner  variables  will  require  considerable,  control. 


STANDARDIZED  SUPERSONIC  TRAINER 

CHECKS 


PERFORMANCE 


Tlic  Supersonic  Trainer  and  the  Student’s  Task 

The  supersonic  trainer  is  a  ground  school  device  which  simulates 
air-borne  navigation  and  bombing  with  radar  equipment.0  The  trainer 
consists  of  a  standard  radar  set,  artificial  terrain  submerged  in  water, 
glass  map,  and  control  panel.  The  antenna  of  the  trainer  transmits 
and  receives  wave  impulses  above  the  frequency  of  sound,  rather  than 
radio  waves  as  in  standard  radar  equipment.  This  antenna  is  moved 
over  the  artificial  terrain  by  a  system  of  electrically  driven  pulleys. 
The  effect  is  almost  identical  to  the  movement  of  an  air-borne  radar 
antenna  over  the  ground.  The  terrain  of  the  trainer  is  made  of  glass, 
sand,  and  carborundum  so  that  returns  from  it,  when  seen  on  tho  radar 
scope,  dnplioato  essentially  returns  from  water,  land,  and  cities.  The 
movement  of  tho  antenna  is  determined  on  tho  basis  of  tho  true  air 
speed,  heading,  and  wind  which  are  regulated  from  a  control  panel. 
It  is  thus  possible  to  present  on  tho  ground,  radar  navigation  and 
bombing  missions  which  arc  strikingly  similar  to  air-borne  missions. 

A  feature  of  the  apparatus  of  particular  value  to  tho  development 
of  standardized  supersonic  performance  checks  is  tho  glass  map  -with 
marking  pen  attachment.  This  map  is  directly  above  the  artificial 
terrain  and  represents  every  feature  of  the  terrain.  A  marking  pen 
follows  the  course  of  the  antenna,  corresponding  to  tho  track  of  an  air¬ 
craft,  making  a  continuous  record  on  paper  stretched  beneath  the 
map.  Thus  a  complete  record  of  the  mission  is  made,  including  course 
corrections  and  bomb  drops. 

Tho  instructional  purpose  of  tho  supersonic  trainer  was  to  teach 
the  use  of  radar  equipment  for  navigation  and  bombing.  In  succes¬ 
sive  training  periods,  tho  task  became  progressively  more  complex, 


'  For  n  I’ullor  description  of  tlio  equipment,  eec  chapter  4. 

106 


developing  from  simple  navigation  to  coordinated  bombing  missions. 
The  student  learned  scoped  interpretation,  fix  taking,  log  procedure, 
j.;XA  computation,  precision  turn  procedure,  drift  correction,  and  tho 
techniqno  of  bond)  release. 

Three  performance  checks  were  constructed  for  the  supersonic 
trainer.  One  was  an  intermediate  navigation  check  given  early  in 
tho  course  after  basic  navigation  techniques  had  been  taught.  Tho 
second  was  an  intermediate  bombing  check  given  after  instruction 
in  coordinated  bombing  procedures.  Third  wn.s  a  final  check  to 
evaluate  the  student's  performance  on  a  combination  navigation  and 
coordinated  bombing  mission  at  the  completion  of  supersonic  train¬ 
ing.  Each  of  tho  tlireo  checks  was  planned  to  require  -10  to  50  min¬ 
utes  per  student. 

Supersonic  Check  Construction 

The  selection  of  critical  items  to  cover  the  tasks  of  the  supersonic 
missions  was  much  more  difficult  than  had  been  the  selection  of  items 
for  the  bench  set  check.  On  the  supersonic  trainer,  it  was  impossible 
to  evaluate  every  aspect  of  behavior  during  an  entire  mission.  Meas¬ 
urement  had  to  be  limited  to  a  sample  of  the  possiblo  performance. 
This  samplo  was  chosen  in  conferences  with  instructional  personnel. 
For  instance,  it  was  agreed  by  training  personnel  that  the  proper  way 
to  start  a  navigation  mission  was  to  take  a  fix  soon  after  departure. 
Since  there  wras  complete  accord  on  this,  since  students  were  taught 
to  do  it,  and  since  it  was  susceptible  to  objective  measurement,  tho 
appropriate  item  was  constructed.  In  this  same  manner,  critical 
items  were  chosen  to  cover  the  remainder  of  the  mission. 


The  Typical  Supersonic  Check 

Disc  ussion  of  the  typical  supersonic  check  will  be  confined  to  t ho 
haul  cheek  since  this  includes  both  the  navigation  and  bombing  items. 
Tho.  typical  final  chock  mission  consisted  of  one  navigation  leg  and 
two  coordinated  bomb  runs.1  The  student’s  task  wns  to  navigate  to 
the  first  initial  point,  make  a  precision  turn  onto  the  bomb  run  head¬ 
ing,  nnd  bomb  the  first  target.  lie  was  then  to  give  a  new  heading 
from  this  target  and  make  a  bomb  run  on  the  second  target.  On  tho 
navigation  leg.  the  student  was  checked  on  17  items  which  evaluated 
ruch  performance  as  his  wind  determination  procedure,  tho  accuracy 
of  the  velocity  and  direction  of  his  computed  wind,  the  accuracy  of  his 
course  correction,  his  log  procedure,  and  the  accuracy  of  his  KT.Y  and 
h-es.  He  was  checked  once  on  each  bomb  run  for  each  of  eight  imr..; 
which  evaluated  the  accuracy  of  his  drift  corrections,  the  accuracy  of 
1  is  announced  sighting  angles,  and  the  accuracy  of  his  bomb  hit. 

The  introductory  material  to  this  check  contained  the  conventional 
remarks  to  the  instructor,  the  statement  of  training  prerequisites,  and 


f  Cor  a  do  script  Ion  of  coordinated  bombing,  bco  diopter  4. 
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tile  <  qnipment  needed  by  student  and  examiner.  The  chnraeterihtir.s 
of  tin*  mi  . ■'ion  were  stilted  and,  more  important,  the  actual  mission  to 
he.  flown  was  defined  in  detail.  Necessary  data  for  the  mission  weiv 
reproduced  on  a  separate  sheet  from  which  the  student,  prepared  h:;< 
flight  jdnn.  In  a  later  form  of  the  check,  the  flight  plan  itself  was 
mimeographed  on  a  special  student  log.  This  saved  time  ami  effected 
further  standardization.  The  student  was  given  altitude,  true  air 
speed,  starting  point,  initial  points,  targets,  and  the  true  courses  .out 
distances  between  these  points.  Since  the  determination  of  the  wind 
was  perhaps  the  most  critical  item,  the  student’s  mission  plan  did  not 
include  wind  velocity  or  direction.  Wind  information  which  was  to 
he  set  into  the  trainer  was  given  to  the  examiner  in  the  directions  for 
administration.  The  student  briefing  section,  which  the  student  rend 
for  himself,  informed  him  of  certain  special  requirements  of  the 
mission. 

The  definition  of  the  standard  mission  in  terms  of  specific  routes 
and  targets  was  essential  to  cheek  standardization.  A  given  item  pre¬ 
sented  an  equivalent  task  to  all  students  only  when  the  raw  material 
of  the  item  task  was  the  same  for  each.  Since  different  areas  varied 
in  the  number  of  usable  radar  returns  presented,  it  was  necessary  that 
the  terrain  variable  be  kept  constant  by  having  all  students  cover  the 
same  or  strictly  comparable  routes.  Similarly,  equivalence  in  length 
of  mission  legs  was  important  since  speed  of  performance  was  a  fac¬ 
tor  in  allowing  opportunity  to  accomplish  all  the  required  items. 

A  refinement  of  the  Inst  form  of  supersonic  check  was  the  inclusion 
of  a  working  plan  for  the  examiner  which  outlined  the  easiest  and  most 
efficient  way  to  administer  the  cheek.  Another  refinement  inaugurated 
the  sub-item,  which  was  expected  to  improve  cheek  administration. 
The  sub-item  was  used  in  any  item  in  which  more  than  one  condition 
had  to  he  suth  fied  for  the  student  to  earn  a  check.  For  such  items  sep¬ 
arate  cheek  spares  were  provided  on  the  scoring  sheet.  This  break¬ 
down  had  two  objectives.  It  eliminated  the  need  for  the  examiner  to 
keep  several  conditions  in  mind  and  it  prevented  him  from  giving 
credit  because  the  student  got  “most  of  the  item”  right.  Clerical  work¬ 
ers  who  totaled  the  cheek  marks  were  instructed  to  count  one  cheek 
for  an  item  only  if  all  of  the  sub-items  were  checked. 

Types  of  Supersonic  Check  Items 

The  items  which  constituted  the  supersonic  checks  were  of  several 
types.  One  major  distinction  is  between  items  which  evaluated  correct 
performance  procedure  mid  those  which  measured  precision  of  per¬ 
formance.  The  procedure  items,  which  comprised  about  one  third  of 
the  items  on  late  forms  of  the  final  check,  were  of  two  kinds.  One  was 
a  simple  observation  of  student  behavior  such  as: 
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Jn!<  iphnnc  promh/rr.  —  fliiH-k  only  if  stndrnt  gives  tlmo  of  fix  and  range(s)’ 
uil/or  ln':iring(s.)  of  targrt  to  navigator  immeilialrly  after  recording  them  In 
’ii;5  log. 

The  other  was  a  simple  examination  of  t ho  student’s  log: 

fjinj  prorrdiu  •;  Ground  spent  unit  KT.  1. —  Clieck  only  If  student  at  time  of 
u,in>e  eorreetion  (see  Item  G) 

(a)  logs  ground  speed. 

( b)  logs  ETA  to  turning  point. 

'I’lie  project  developed  several  devices  to  aid  in  evaluating  precision 
on  the  trainer.  One  of  these  was  the  method  of  marking  fixes  on  tlio 
gln-s  map  so  that  they  could  be  compared  accurately  with  the  corres¬ 
ponding  plotted  fix  on  the  student’s  chart.  This  was  accomplished  by 
means  of  the  bomb  release  switch  which,  on  a  direct  bomb  run,  caused 
the  marking  pen  to  fall  away  from  the  glass  map  and  thus  break  tho- 
inl;  line.  On  a  bomb  run  the  pen  did  not  mark  again  until  the  trainer 
.-topped  after  the  time  of  fall  of  the  bomb  bad  run  out.  However,  if 
the  reset  switch  ou  the  control  panel  was  pushed  before  the  timo  of  fall 
ran  out,  the  pen  would  continue  marking  at  once,  and  the  trainer  would 
not  stop.  Therefore,  the  student  was  briefed  to  push  the  bomb  release 
switch  at  the  instant  be  took  a  fix  which  he  planned  to  plot,  lie  was  to 
call  ‘‘fix”  simultaneously  to  warn  the  examiner  to  push  the  reset  switch 
about  10  seconds  later.  Thus  every  fix  was  marked  by  n  break  in  tho 
ink  track  line  on  the  glass  map.  On  models  of  the  trainer  not  equipped 
with  a  bomb  release  switch,  the  student  simply  called  “fix”  and  tho 
examiner  produced  the  break  in  the  ink  track  line  by  use  of  the  record¬ 
ing  pen  switch  on  the  control  panel. 

Another  aid  to  precision  measurement  developed  by  the  project 
was  the  supersonic  plotter.  The  first  and  most  simple  plotter  was- 
a  wide,  plexiglass  straight-edge  on  which  several  scales  were  etched.3 
f’m*  scale  along  the  edge  was  calibrated  in  glass-map  nautical  miles 
ro  that  distances  would  bn  read  directly  on  the  glass  map  itself.  An¬ 
other  scale  consisted  of  a  length-wise  center  line  with  various  parallel 
lines  so  that,  by  placing  the  center  line  over  any  course,  distance  of 
the  track  away  from  that  course  could  be  read  directly.  Two  scales 
of  concentric  nautical  mile  circles  allowed  for  reading  distance  away 
I  com  any  point  over  which  the  center  of  the  circle  was  placed. 

the  latest  plotter  consisted  of  a  square  plexiglass  base,  etched  with- 
a  protractor  scale,  and  bearing  two  rotatable  plexiglass  arms.9  Y/ith 
this  device,  fixes  could  he  transferred  from  the  student’s  chart  to  tho 
.'•lass  map  without  computation  or  additional  mechanical  aids..  Am- 
gnlnr  differences  between  desired  course  and  actual  track  could  also 
hr>  determined  easily.  Other  scales  etched  on  the  square  base  fac'd i- 
h  tod  the  measurement  of  ETA  and  bomb  hit  accuracy. 

’  Ohl t  plotter  v.-.-n  (I, signed  by  Sgt.  Mancnn  anil  con.strucled  by  T/Sgt.  George  N. 
Eeillnger. 

‘This  plotter  was  designed  by  Cpl.  Kelley  and  conntructed  by  T/Sgt.  Bollinger. 
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To  make  1 1  Hi  measurement  of  pivci-ion  a t  all  feu  ible  it  was  nec¬ 
essary  In  taka  account  of  ceitain  marcuracie;  in  the  truirvr.  For 
example,  tin;  wind  which  tin.*  trainer  produced  did  not  always  cor- 
respond  exactly  to  the  wind  set  in  the  control  panel.  Coivwqueetly, 
since  the  student  was  graded  on  the  accuracy  of  his  '•nmputaton  of 
wind  direction  and  force,  a  heller  standard  than  the  wind  set  in  the 
trainer  was  needed.  A  more  accurate  standard  was  provided  hy  hav¬ 
ing  the  examiner  determine  ground  speed  and  track  !>;,  measurement 
on  tin!  glass  map.  By  combining  these  data  with  the  true  air  speed 
and  heading  given  to  the  .student  in  the  mi  sion  plan,  the  examiner 
could  determine  the  wind  which  the  trainer  was  actually  producing. 

Precision  items  accounted  for  about  two  thirds  of  the  items  on  the 
latest,  forms  of  the.  final  supersonic  cheek.  Such  items  ma}'  be  grouped 
into  rcveral  types.  One  type  is  a  simple  comparison  of  data  logged 
by  the.  student  with  comparable  data  set.  into  the  trainer.  A  second 
type  involved  an  observation  of  behavior  plus  a  simple  computation. 
For  example,  the  accuracy  of  tho  student’s  multiple  drift  correction 
was  determined  by  having  the  examiner  noto  all  tile  course  corrections 
given  by  (lie  student  after  the  multiple  drift  correction  and  add  them 
algebraically  to  see  if  they  totaled  4°  or  less.  This  item  illustrates 
also  indirect  measurement  of  performance  in  that  measurement  was 
applied  to  subsequent  behavior  causally  related  to  tho  behavior  for 
which  evaluation  is  desired.  In  this  instance  the  accuracy  of  tho  mul¬ 
tiple  drift  correction,  which  is  the  point  of  concern,  could  bo  inferred 
from  the  subsequent  correction  necessary  to  bring  the  aircraft  over 
the  target.  A  third  type  of  evaluation  of  precision  was  accomplished 
by  measurements  on  tho  glass  map.  In  evaluating  tho  bombing  de¬ 
flection  cr:  for  example,  tho  examiner  laid  tho  plotter  along  tho 

inked  track  at  “Bombs  Away”  and  determined  whether  tho  track 
or  its  extension  came  within  one.  nautical  mile  of  the  briefed  aiming 
point.  A  fourth  type  of  precision  item  compared  the  inked  track 
record  on  the  glass  map  with  the  student’s  chart  work.  In  such  items 
the.  nernraey  of  the  student’s  fixes  was  determined  by  transferring 
the  fix  data  from  the  student’s  chart  to  the  glass  map  by  means  of 
the  protractor  and  determining  whether  his  fix  was  accurato  within 
the  stated  tolerance.  A  final  type  required  the  examiner  to  jndgo 
the  quality  of  the  student’s  performance,  and,  if  necessary,  to  compare 
it  with  his  own.  In  evaluating  set  operation,  for  instance,  tho  ex¬ 
aminer  was  to  read  range  and  naimuth  while  the  student  was  taking 
a  fix  and  judge  whether  the  scope  returns  were  defined  well  enough 
to  make  accurate  readings  possible.  If  he  thought  they  were  not,  ho 
was  to  adjust  the  radar  set  himself  to  determine  whether  better  defi¬ 
nition  was  possible.  In  later  revisions  of  the  supersonic  checks,  there 
were  no  items  which  depended  upon  this  degree  of  examiner  judgment. 
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For  most  precision  items,  such  as  the  accuracy  of  wind  computation, 
it  was  necessary  to  establish  acceptable  limits  or  tolerances.  The  tol- 
ci'a nees  for  the.  earliest  form  of  the.  final  supersonic  check  were  based 
upon  a  study  of  the  moan  errors  of  a  trial  group  of  10  radar  students 
taking  a  preliminary  version  of  the.  check.  Later  forms  included  tol¬ 
erances  1*. i  ed  on  more  precise  study  of  item  difficulty  levels. 

Supersonic  Check  Administration 

Difficulties  in  maintaining  standard  administration  of  the  super¬ 
sonic  checks  were  numerous.  Variability  in  equipment  accounted  for 
many  of  these.  As  noted  above,  data  set  into  the  trainer  did  not 
always  produce  the  expected  result.  Most  of  this  difficulty  was  com¬ 
pensated  for,  however,  by  having  the  examiner  compute  the  wind  from 
the.  trainer  in  the  manner  already  described.  The  recording  pen  some¬ 
times  failed  to  mark  or  ran  out  of  ink  during  the  mission,  making 
items  to  be  measured  from  the  glass  map  difficult  or  impossible  to 
evaluate.  On  other  occasions,  the  pen  marked  adequately  but  failed 
to  drop  so  as  to  mark  fixes  and  ETA’s.  The  examiner  was  forced  to 
use  his  fountain  pen,  if  he  had  one,  and  lost  valuable  time  in  marking 
by  hand.  Some  trainers  tended  to  stick  on  cardinal  headings  which 
confused  the  student  and  made  the  track  line  an  inaccurato  index  of 
the  student’s  intention.  The  most  persistent  trainer  difficulty  was 
the  misalignment  of  the  glass  map  with  the  artificial  terrain.  If  tho 
difference  between  the  two  was  great,  it  was  clearly  unfair  to  take  tho 
glass  map  record  as  indicative  of  tho  student’s  performance.  Many 
attempts  were  made  to  deviso  a  method  for  checking  exactly  tho 
alignment  of  map  and  terrain,  hut  no  solution  had  been  obtained  by 
the  time  work  on  tho  checks  was  discontinued.  Only  gross  differences 
wero  readily  recognized  and  corrected. 

Examiner  variability  accounted  also  for  lack  of  standardization. 
As  on  the  bench  check,  tho  halo  effect  was  in  evidence.  In  addition, 
inadequate  student  briefing  occasionally  resulted  in  tho  student  pro¬ 
ceeding  with  the  mission  without  sufficient  knowledge  of  what  was 
required  of  him.  Also,  some  examiners  failed  to  check  each  item  as 
it-  was  performed,  and  scoring  by  memory,  was,  of  course,  unreliable. 
Other  examiners  estimated  error  instead  of  carrying  out  the  more  exact 
measurements  required  by  the  administrative  directions.  Finally, 
some  examiners  seemed  to  identify  with  tho  student  and  could  not 
avoid  hints  and  leading  questions  which  helped  the  student  correct 
what,  they  regarded  as  “foolish”  errors. 

Supersonic  Check  Revisions 

Four  forms  of  the  Final  Supersonic  Check  wero  published. 
One  (P7a-A)  was  used  at  Langley  Field  and  Boca  Raton  for 
tho  AN/ A  PS-15,  AN/APS-15A,  and  AN/APQ-13."  Another 

”  Snpi  rsonlc  Cheek  P7n-A  was  constritf toil  by  pRt.  Wllllnm  J.  n ml  Sjjt.  Sheldon 

U.  NVrby  under  the  supervision  of  Cnpt.  Horace  H.  Vun  Faun. 
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(L  l’7;i  B),  used  at  Langley  Field,  w;t,  al.-o  :i ppl ill*  to  each  of 
t hose  Oik*  (V  P7a-A)  was  u  ed  to  check  pi olicmin-y  with  tlu> 

AX/APQ-13  <*« ji s 1 1 >1  iii-t 1 1  in  tlic  Wc-tcni  Flying  Training  Command 
while  tin*  fourth  ( W  F7F-A )  was  used  at  Williams  Field  to  evaluate 
performance  with  the  AX/AFQ-7.'-  The  two  most  recent  forms  of 
the  filial  supersonic  check  incorporated  numerous  improvements  and 
additions.  Appendix  II  includes  one  of  these,  the  last  in  use  at 
Langley  Field.  Tho  other,  prepared  for  use  with  AN/APQ-13  in 
the  Western  Flying  Training  Command,  is  difleient  in  certain 
respects.  In  addition  to  requiring  the  checking  of  sub-items,  as 
described  earlier,  it  reduces  instructions  to  l.  examiner  to  the 
minimum  necessary  for  checking  typical  performance.  Evaluation 
of  a  typical  performance  was  described  in  a  section  of  the  booklet 
called  Unusual  Situations.  This  variation  in  format  represented  an 
adjustment  to  a  recurrent  problem  encountered  in  the.  preparation  of 
material  for  check  administrators  who  were  unsophisticated  in  the 
requirements  of  standardized  measurement.  It  was  necessary  con¬ 
stantly  to  compromise  between  the  advantages  of  exhaustive  detail  in 
administrative  instructions  and  the  tendency  observed  in  examiners 
to  he.  confused  uml  antagonized  by  such  detail. 

Six  forms  of  the  Radar  Navigation  intermediate  Supersonic  Check 
wero  produced.  P7e-A  and  P7c-B  were  used  in  conjunction  with  the 
AN/APS-15  and  AN/APS-15A  at  Langley  Field,  and  tho  AN/APQ- 
13  at  Boca  Raton.  Form  L  P7e-C,  applicable  to  these  same  sets,  was 
used  at  Langley  Field.  Form  V  P7c-A  ( AN/APQ-13)  was  developed 
for  the  Western  Flying  Training  Command.  Foitii  W  P7o-A  was 
a  temporary  form  used  at  Williams  Field  with  tho  AN/APQ-13  set 
Form  W  1*7 f-A  measured  navigation  skill  on  the  AN/APQ-7  equip¬ 
ment.13 

Three  forms  of  the  Radar  Bombing  Intermediate  Supersonic  Check 
were  developed.  Form  P7e-A,  used  for  AN/APS-15  and  AN/APS- 
15A  and  AN/APQ-13,  was  administered  at  Langley  Field  and  Boca 
Raton.  Form  L  P7c-B,  usable  on  all  three  sets,  was  given  at  Langley 
Field.  Form  V  P7c-A  was  used  with  the  AN/APQ-13  in  tho  Western 
Flying  Training  Command.14 

Supersonic  Cheek  Statistical  Findings 

Figures  0.2,  0.3,  and  0.4  present  diagrams  of  typical  distributions 
of  semes  on  the  navigation,  bombing,  and  final  check,  respectively. 

11  Sgt.  Miuu.mii  developed  this  chock. 

”  Ut.  Orcir^.'  S.  Kl.'ln  iiiid  C|'l.  Bray  with  tip*  arelHtnnce  of  Sgt.  Nerby  const rnctcd  fona 
V  I’Ta  A.  while  form  W  1*71) -A  was  developed  by  C* | >1 .  Urny. 

”  S|?t.  Mnm.Mii  nml  Set.  N'erl>y  wrr.*  responsible  for  I lie*  development  of  three  of  the  si* 
Radar  Navigation  1  n 1 1- r jii> •*! I :i I <•  Sii|..  r.-n:ile  Cheeks.  The  oilier  three  ch.-ckri  were  con- 
si  ro  tel  hy  r.t  Klein.  Ci-1.  Urny.  nml  C|>1.  nnrohl  II.  Kelley. 

“  Chief  coiitrlhiilorx  to  the  development  of  the  Radar  Itomblng  Intermediate  Supersonic 
■Clucks  were  Sgt.  Mnitcnn,  Sgt.  Nerby,  Cpl.  Itray»nnd  Lt.  ICleln. 
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F ilium:  G.2 — Frequency  distribution  of  rnw  scores  for  n  standardized  radar  navi¬ 
gation  intermediate  supersonic  check  (P7e-CX) — administered  at  Langley 
Field. 


Tho  navigation  check  had  a  possible  maximum  raw  score  of  37.  Tho 
mean  score  of  22.8  indicates  that  the  check  was  sufficiently  difficult. 
The  distribution  is  not  significantly  skewed.  The  bombing  check  had 
a  total  maximum  raw  score  of  40.  The  mean  score  of  34.2  and  tho  fact 
that  many  students  made  the  highest  possible  score  shows  that  this 
‘■heck  was  too  easy.  The  distribution  is  significantly  skewed  and  lias 
•n  narrow  range.  The  final  check,  combining  both  navigation  and 
bombing  items,  had  a  possible  maximum  raw  score  of  54  and  a  mean 
score  of  43.0.  This  distribution  is  also  significantly  skewed. 

The  product  moment  correlation  between  check  scores  on  tho  first 
:md  second  legs  of  the  navigation  check  was  0.3G1  (N=1S9).  Tho 
correlation  between  scores  on  the  first  and  third  legs  ns  against  tho 
,f,ond  and  fourth  legs  of  the  bombing  check  was  0.1GG  (N  —  190). 
ho  first  and  second  bomb  runs  of  the  final  check  correlated  0.800 


51G).  These  coefficients,  while  of  interest,  are  not  presented  as 
indices  of  reliability.  The  only  acceptable  measure  of  reliability  on 
A  ‘'heck  of  this  sort  is  a  tcst-rctcst  coefficient.  Tho  modified  split  half 
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N  ■  576  Skew3— 1.50 

Mean®  34.18  SDaK*  .24 

SD  «  4.81  C.R.5K®  6.25 

FiouitK  0.3. — Frequency  distribution  of  raw  scores  for  a  standardized  radar  bomb¬ 
ing  intermediate  supersonic  check  (P7c-A) — administered  at  Langley  Field. 

method  which  yielded  tho  correlations  given  abovo  is  not  acceptable 
becauso  tho  samo  examiner  administered  both  parts  of  tho  check  and 
testing  conditions  wero  tho  samo  for  both  halves. 

A  test-retest  reliability  study,  the  only  one  attempted,  resulted  in  a 
coefficient  of  0.49  (N  — 79).  This  is  significant  at  the  1-percent  level, 
tho  probablo  error  of  a  zero  correlation  based  on  79  cases  being  0.076. 
Thoexporiincnfc  was  unstandardized,  in  part, because  of  VJ-day,  which 
was  announced  near  the  completion  of  tho  check  administrations.  It 
is  feared  that  this  modified  tho  usual  student  and  examiner  behavior. 
Provious  cases  show  a  first  administration  mean  of  32.19  and  a  recheck 
mean  of  33.05  (N  =  55).  Tho  corresponding  data  for  cases  admin¬ 
istered  after  VJ-day  wero  31.42  and  35.25  (N=24).  It  was  recog¬ 
nized  during  tho  experiment  that  tho  examining  board  which  admin¬ 
istered  tho  checks  wero  not  motivated  to  provide  well-standardized 
administration.  Consequently,  relatively  low  reliability  was  ex¬ 
pected.1' 


11  C.ipb  Onbrlol  D.  Oflosli  supervised  this  reliability  study.  Do  was  assisted  by  S/SeC 
Bormml  C.  Sullivan  and  Sgt.  Mnngnn. 
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Table.;  GT>  ami  G.  1  show  examiner  means  and  sigmas  for  final  supor- 
(  becks  /oven  at  Langley  Field  and  at  Boca.  Raton,  Again  it  was 
inijx'-ssiblc  to  use  the  analysis  of  variance  technique  since  a  chi-square 
u-.  t  uf  the  Langley  data  for  homogeneity  of  variance  resulted  in  a 
\ahui  of  9d,  being  needed  for  significance  at  the  1-percent  level. 
The  same  test  on  (lie  Boca  Raton  figures  yielded  a  value  of  37,  20.09 

Taiiu:  C.3.— Examiner  means  and  standard  deviations  for  a  standardized  final 
tupersonw  truhicr  performance  check  (P7a^A)  administered  at  Langley  Field 
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i  The  highest  posable  raw  score  on  this  performance  check  was  at 
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sonic  check  (P7a-A)  — fulminwlcrcd  nt  Langley  Field. 
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Tari.k  fl.4. — Examiner  means  and  standard  deviations  for  a  standardized  final 
supersonic  trainer  performance  cheek  (I‘7a-A)  administered  at  Boca,  Raton 
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*  Tlie  highest  possible  raw  score  on  this  performances  check  was  62. 


being  needed  for  significance  at  the  1-pcreent  level.  Both  sets  of  data 
reveal  wide  variation  in  means  and  sigmas.  With  the  same  qualifica¬ 
tions  as  were  stated  in  the  similar  discussion  of  the  bench  set  data,  it 
may  be  concluded  that  standardized  administration  was  not  achieved. 

STANDARDIZED  AERIAL  PERFORMANCE  CHECKS 

The  Airborne  Set  and  the  Student’s  Task 

The  most  heavily  weighted  and  academically  important  perform¬ 
ance  checks  were  those  developed  to  evaluate  the  student’s  proficiency 
as  a  radar  observer  under  actual  flight  conditions.  Apparatus  in¬ 
volved  in  the  administration  of  these  checks  was  the  air-borne  radar 
set  and  auxiliary  flight  instruments.  Briefly  stated,  the  radar  ob¬ 
server’s  task10  was  to  direct  the  aircraft  on  the  briefed  course  to  the 
initial  point  and  make  an  accurate  bomb  run  on  the  briefed  target. 
To  this  end,  the  objective  of  training  was  to  produce  students  able 
to  operate  the  set;  to  compute  accurate  winds,  course  corrections, 
ETA’s  and  other  navigational  data;  to  kill  drift,  determine  absolute 
altitude,  and  bomb  accurately  in  collaboration  with  the  bombardier; 
and  to  keep  a  complete  and  accurate  log. 

Two  aerial  performance  checks  were  developed:  the  intermediate 
aerial  check  given  mid-way  in  aerial  training  after  the  essentials  of 
radar  navigation  and  bombing  had  been  taught,  and  the  final  aerial 
check  given  at  the  completion  of  the  course.  The  latter  check  will  be 
discussed  first. 

Aerial  Check  Construction 

Tho  method  of  making  the  original  selection  of  items  for  the 
aerial  check  was  similar  to  that  outlined  above  in  the  discussion  of 
tho  supersonic  check.  Project  personnel  took  numerous  flights  to 
niuko  systematic  observations  of  the  task  of  the  radar  observer,  en¬ 
rolled  in  the  radar  course,  and  worked  in  conference  with  traimn;,' 
authorities.  Then  followed  the  selection  of  critical  aspects  of  radar 
observer  performance  on  an  aerial  mission.  Developing  a  method 

31  For  a  Job  analjolB  of  tho  radar  observer,  see  cli.  i. 
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of  measurement  for  these  critical  items  was  more  difficult  hero  than  for 
other  performance  checks  because  of  the  difficulty  of  finding  objective 
criteria  of  success. 

The  Typical  Aerial  Check 

The  Aerial  checks  contained  the  conventional  introductory  sec¬ 
tions  previously  described.  The  introductory  material  included  gen¬ 
eral  comments  on  performance  checks,  a  listing  of  the  equipment 
needed  by  student  and  examiner,  the  training  prerequisites,  detailed, 
instructions  for  administration,  and  the  characteristics  oi  the  stand¬ 
ard  mission.  The  aerial  checks  also  had  a  section  containing  sup¬ 
plementary  instructions  for  each  item;  this  followed  the  answer  sheet. 
These  instructions  described  the  purpose  of  tho  item,  how  to  check  the 
item,  and  special  situations  which  might  arise  while  checking  the  item.. 
The  list  of  items  was  printed  on  cut-back  pages  with  matching  answer 
sheet. 

The  description  of  the  standard  mission  was  used  to  select  specific 
routes  over  which  the  check  mission  was  required  to  bo  flown.  A 
standard  mission  was  necessary  for  aerial  checks  for  reasons  already 
stated  in  the  discussion  of  supersonic  trainer  performance  checks. 
If  anything,  a  standard  mission  was  needed  even  more  in  tho  aerial 
situation  where  other  variables  such  as  weather  and  turbulence  were 
difficult  to  control. 

The  final  aerial  check  required  for  its  administration  approximately 
five  hours,  a  complete  mission  consisting  of  four  or  five  navigation 
legs  each  follwcd  by  a  bomb  run.  The  intermediate  check  required 
2y2  hours  and  included  cither  two  or  three  navigation  and  bombing 
legs. 

Types  of  Aerial  Check  Items 

The  items  comprising  the  aerial  check  can  be  divided,  as  in  the- 
checks  previously  discussed,  into  tho  broad  categories  of  those  which, 
measure  procedure  and  those  which  measure  precision  of  perform¬ 
ance.  Procedure  items,  winch  accounted  for  about  one-tliird  of  final 
check  items,  were  of  two  types.  One  of  these  types  called  for  a  simple 
oh  serration  of  student  behavior.  For  example,  in  evaluating  per¬ 
formance  of  the  preoperational  check,  the  examiner  merely  noted 
whether  the  students  adjusted  certain  designated  controls  in  tho  proper 
directions.  The  second  type  required  only  an  examination  of  the 
student’s  log  or  chart.  For  example,  in  determining  whether  tho 
student  had  taken  fixes  with  sufficient  frequency,  the  examiner  was 
"cquired  to  inspect  the  log  and  chart  and  from  them  determine  whether 
nu,re  than  15  minutes  had  elapsed  between  successive  fixes. 

The  p  revision  items  amounting  to  about  two-thirds  of  tho  check,, 
were  of  four  types.  One  type  involved  an  observation  of  behavior 
fhis  a  simple  computation.  Such  an  item  was  the  evaluation  of  tli& 
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accuracy  of  the  student’s  course  correction.  In  part,  the  task  of  the 
examiner  was  to  note  and  add  algebraically  (he  changes  in  heading 
which  the  student  called  to  the  pilot  after  making  the  course  correction 
and  before  reaching  the  turning  point  near  tho  initial  point.  A  sec¬ 
ond  type  of  precision  item  was  based  on  ar>  evaluation  of  objective 
results  but  depended  on  the  accuracy  of  the  examiner’s  observation. 
For  example,  the  further  condition  for  receiving  credit  for  an  accurate 
course  correction,  (see  sample  item  above)  was  that  the  aircraft 
pass  within  2  miles  of  the  correct  turning  point  near  the  initial  point. 
This  required  the  examiner  to  compute  the  correct  turning  point  in 
relation  to  the  radar  return  representing  the  initial  point  and  deter¬ 
mine  by  measurement  on  the  scope  how  close  the  aircraft  came  to  this 
point.  A  third  type  of  item  used  for  evaluating  precision  of  per¬ 
formance  required  a  comparison  of  the  student’s  performance  with 
that  of  the  examiner.  The  accuracy  of  the  multiple  drift  correction, 
for  example,  was  evaluated  by  having  the  examiner  determine  the 
correction  simultaneously  with  the  student.  He  was  to  give  credit 
only  if  tho  student’s  correction  was  within  !>°  of  his  own  estimate. 
Tho  fourth  type  of  precision  item  was  based  on  the  examiner’s  judg¬ 
ment.  For  example,  in  evaluating  the  student’s  skill  in  operating  tho 
set  so  as  to  define  adequately  the  aiming  point  on  the  scope,  the  exam¬ 
iner’s  task  was  to  judge  whother  tho  definition  was  the  best  possible 
within  tho  limits  of  tho  set.  Only  a  single  item  of  this  nature  was 
included  in  the  check. 

Tho  intermediate  check  differed  slightly  from  tho  final  check  at  two 
of  tho  radar  schools.  Besides  being  a  shorter  check,  as  previously 
stated,  the  first  few  items  were  oral  questions  concerning  trouble  shoot¬ 
ing  and  the  location  of  certain  units  of  the  equipment.  In  addition, 
tho  bornb  runs  were  by  direct  rather  than  by  coordinated  procedure. 
In  tho  other  schools  tho  intermediate  check  was  identical  to  vue  final 
check  except  that  it  was  given  on  two  instead  of  four  legs. 

Aerial  Cheek  /id ministration 

Standardization  of  aerial  check  administration  was  most  difficult 
Tho  equipment,  including  tho  aircraft,  accounted  for  some  of  the  vari¬ 
ation.  Various  malfunctions  wero  likely  to  develop  during  check  ad¬ 
ministration.  Since  these  were  frequently  not  serious  enough  to  war¬ 
rant  returning  to  tho  base,  the  mission  was  completed.  In  such  cases, 
it  was  impossible  to  determine  precisely  how  much  the  malfunction 
interfered  with  tho  student’s  performance.  Ono  such  equipment  dif¬ 
ficulty  was  that  of  varying  range  of  tho  radar  set.  One  set  might 
show  returns  from  80  miles  away,  another  might  “get”  only  50  miles. 

Weather  was  another  condition  of  check  administration  which  could 
not  bo  standardized.  Bad  wcathor  on  tho  briefed  courso  involved  fly¬ 
ing  around  it;  thunderheads  interfered  with  radar  roturns,  and  more 
than  usual  turbulence  affected  performance. 
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Pilot  viiriiition  was  greater  than  might  he  expected.  Some  pilots 
wore  conscientious  and  followed  interphone  instructions  from  the 
radar  observer  quickly  and  accurately.  Others  were  not  as  coopera¬ 
tive.  a  fact  which  markedly  hindered  the  student's  work.  Delay  by  the 
pilot  in  making  a  course  correction  on  the  bomb  run,  for  example,  might 
spoil  what  otherwise  would  have  been  u  good  performance. 

Variation  in  altitude  also  resulted  in  some  lack  of  standardization. 
Although  pilots  were  briefed  for  a  given  check  flight  altitude,  this  in¬ 
struction  was  not  always  carefully  followed.  Differences  in  altitude 
changed  the  difficulty  of  some  items.  For  example,  synchronizing 
with  the  bombardier  on  the  bomb  run  is  a  more  leisurely  matter  at 
20,000  feet  than  at  10,000  feet  because  the  sighting  angles  follow  one 
another  more  slowly.  Altitude  variation,  moreover,  required  some 
students  to  work  in  an  oxygen  mask  while  others  were  not  so  encum¬ 
bered. 

In  an  effort  to  take  account  of  variations  which  practical  circum¬ 
stances  made  it  impossible  to  control  administratively,  an  addition  was 
made  to  the  scoring  procedure.  If  the  student  failed  to  meet  the  stand¬ 
ards  prescribed  for  an  item,  regular  practice  called  for  the  examiner 
to  enter  a  zero  in  the  scoring  space  regardless  of  whether  or  not  the 
failure  was  for  reasons  beyond  the  student’s  control.  However,  if 
the  examiner  believed  that  the  student’s  performance  on  a  failed  item 
was  as  satisfactory  as  it  could  he  under  adverse  circumstances,  he  was 
now  instructed  to  enter  an  “S”  with  the  zero.  Obviously,  this  attemp¬ 
ted  compensation  for  nonstandard  conditions  injected  relatively  un- 
contiollod  examiner  judgment  into  a  check  designed  to  eliminate  it. 
To  offset  this  weakness,  it  was  provided  that  all  “S”  scores  would  be 
reviewed  by  the  chief  examiner.  To  accumulate  experience  for  stand¬ 
ardizing  bis  judgment,  the  chief  examiner  was  directed  to  determine 
exactly  the  conditions  loading  to  each  “S”  score  and  keep  a  record  of 
which  of  these  conditions  warranted  the  giving  of  credit  for  the  item. 
To  aid  in  this  process,  it  was  arranged  that  each  examiner  was  to  keep 
a  mission  log  on  which  lie  noted  the  reason  for  each  “S”  score. 

As  on  the  other  performance  checks,  the  examiners  themselves  were 
responsible  for  departures  from  standard  administration.  'The  halo 
effect,  hints,  and  leading  questions  were  in  evidence  here  as  elsewhere. 
Failure  to  mark  items  as  they  were  performed  resulted  in  further 
inaccuracies.  One  reason  for  poor  administrations  of  aerial  checks 
was  the  difficulty  which  the  examiners  had  in  obtaining  the  data  in 
terms  of  which  student  performance  was  checked.  Many  of  tluvo 
data  bad  to  be  taken  from  the  scope  itself.  This  meant  that  both 
student  and  examiner  had  to  use  the  same  scope,  frequently  nt  the 
same  time.  This  difficulty  was  partially  solved  by  blacking  out  the 
radar  observer’s  com  part  incut-  so  that  the  hood  on  the  scope  could  be 
removed.  Under  those  conditions  student  and  examiner  could  watch 
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tho  returns  simultaneously  although  the  examiner  l  ad  to  move  fre¬ 
quently  to  avoid  errors  due  to  parallax.  In  eases  where  the  com¬ 
partment  was  not  blacked  out  and  the  hood  had  to  be  used,  the 
examiner  had  the  uncomfortable  task  of  leaning  over  the  student’s 
shoulder  and  taking  readings  as  best  lie  could.  When  the  discomfort 
of  heavy  flight  clo  hing  and  oxygen  mask  is  added,  the  exacting  and 
strenuous  nature  of  the  job  of  an  examiner  on  a  5-hour-check  ride 
can  be  appreciated. 

Aerial  Cheek  Revisions. 

Only  two  forms  of  the  Intermediate  Aerial  Performance  Cheek  were 
necessary.  One  (P8b-A)  was  used  to  evaluate  aerial  performance 
with  the  AN/APS-15  and  AN/APS-15A  at  Langley  Field  an  tho 
AN/APQ-13  at  Boca  Raton.  The  other  (V  PSb-A)  was  used  with 
tho  AN/APQ-13  in  the  Western  Flying  Training  Command. 

Four  forms  of  the  Final  Aerial  Performance  Cheek  were  prepared. 
Two  (P8a-A  and  P8a-B)  were  applicable  to  the  AN/APS-15  and 
AN/APS-15A  at  Langley  Field  and  the  AN/APQ-13  at  Boca  Raton. 
Another  (V  P8a-A)  w’as  used  with  AN/APQ-13  in  tho  Western 
Flying  Training  Command.  The  fourth  (W  P8c-A)  was  used  to 
evaluate  performance  with  the  AN/APQ-7  sot  at  Williams  Field.17- 

Aerial  Cheek  Statistical  Findings 

Figures  G.5  and  G.G  present  typical  distributions  of  scores  from  the 
intermediate  and  final  aerial  checks.  The  intermediate  check  shows 
a  mean  raw  score  of  41.9  out  of  a  maximum  possible  score  of  G2.  The 
distribution  of  check  scores  is  skewed  significantly.  The  final  check 
yielded  a  mean  of  59.4  out  of  a  maximum  possible  score  of  7G.  Skew 
for  this  distribution  is  not.  statistically  significant. 

Tables  G.5  and  G.G  present  examiner  means  and  sigmas  for  final 
aerial  performance  checks  administered  at  Langley  Field  and  Boca 
Raton.  Here  again  it  was  impossible  to  use  the  analysis  of  variance 
technique.  A  chi-square  test  for  homogeneity  of  variance  of  the 
Langley  data  resulted  in  a  critical  ratio  against  homogeneity  signifi¬ 
cant  at  the  1  percent  level.  For  the  Boca  Raton  data  a  chi-square 
equalled  G9  with  only  27.G9  needed  for  significance  at  the  1  percent 
level.  Both  tables  reveal  largo  differences  in  examiner  means  and  a 
surprising  variation  in  standard  deviations. 

A  correlation  between  the  score  made  on  the  first,  third,  and  fifth 
logs  of  the  final  aerial  check  and  tho  second  and  fourth  legs  yielded 
a  coefficient  of  0.1GG  (N  —  90) .  For  reasons  already  given  in  discussing 
the  supersonic  checks,  care  should  be  taken  not  to  interpret  such  figures 
as  measures  of  reliability. 

1fEnrly  developmental  work  on  Aerial  Performance  Checko  was  performed  by  Capt. 
Von  Soun  and  R/Sgt,  Mitchell  with  the  nsslatance  of  Sgt.  IMum  and  Sgt.  Gnge.  Later 
checku  were  constructed  by  Sgt.  Philip  II.  Krledt,  Sgt.  Gage,  Cpl.  Kelley,  Cpl.  Pray,  and  Lt. 
Klein. 
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Figure  G.5.—  Frequency  distribution  of 
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Taiu.k  0.5. — Examiner  means  ana  standard  deviations  for  a  standardized  final 
aerial  performance  check  (Ptia-Jl)  administered  at  Lamjtcy  Field 
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>  The  highest  possible  row  score  on  this  performance  check  was  88. 


T.Mir.K  0.0. — Examiner  means  and  standard  deviations  for  a  standardized  fin 
aerial  performance  cheek  (I’Ha-ll)  administered  at  flora  Eaton 
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1  The  highest  possible  score  on  this  performance  check  was  81. 


The  Student  Haling  Research  Form  18 

An  addition  to  the  litter  form  of  the  aerial  cheek  was  the  Student 
Hating  Research  Form.  The  first  part  of  the  form  provided  space  for 
the  examiner  to  describe  variables  which  he  felt  influenced  the  student’s 
performance  check  score.  Spaces  were  provided  for  set  malfunction, 
weather  conditions,  crew,  and  other  variables.  The  second  part  con¬ 
sisted  of  four  rating  scales.  The  examiner  rated  the  student  on  set 
operation,  navigation  proficiency,  bombing  proficiency,  and  over-all 
proficiency.  The  examiner  rated  the  student  in  relation  to  all  the  stu¬ 
dents  to  whom  he  had  given  performance  cheeks  and  on  the  basis  of 
how  well  he  thought  the  student  would  have  done  under  “normal  con¬ 
ditions.” 

Sample  items: 

First  part. — Did  the  crew  interfere  with  the  student  or  give  him  help 
when  they  shouldn’t  have?  Explain. 

”  Cpl.  Kelley  developed  (he  Student  Rating  Research  Form. 
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Second  pact. — 1.  Sot  operation  :  Quality  of  scope  picture  the  student 
maintained  through  the  mission. 
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Tho  purpose  of  the  Student  Rating  Research  Form  was  to  make  an 
initial  investigation  into  the  possibility  of  using  examiner  comment 
and  evaluation  to  counteract  some  of  the  uncontrolled  variability  in 
the  administration  of  the  aerial  performance  checks.  Aerial  checking 
was  terminated  before  this  possibility  could  be  tested  adequately,  but 
statistical  findings  on  the  few  cases  at  hand  arc  of  interest.  Computa¬ 
tions  were  made  only  for  the  over-all  proficiency  rating.  At  Langley 
Field  GO  students  received  both  check  scores  and  proficiency  ratings  on 
the  final  aerial  check.  The  coefficient  of  correlation  between  these  two 
values,  as  shown  in  table  0.7,  was  0.505.  The  same  correlation  at  Vic- 


Tabi.e  0.7. — Product-moment  correlations  between  raw  scores  and  instructor 
ratings  on  aerial  performance  checks 
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torville  on  333  cases  was  0.517.  Similar  studies  on  the  intermediate 
check  yielded  at  Langley  Field,  0.777  (N  =  49),  and,  at  Victorville, 
0.GS2  (N  =  200).  It.  must  be  remembered  that  in  this  study  tho  in¬ 
structor  made  his  rating  immediately  after  administering  tho  check, 
and  it  is  reasonable  to  assume  that  his  judgment  was  influenced  by  tho 
student’s  success  on  the  check.  Consequently,  it  cannot  be  said  with 
certainty  that  objective  check  scores  and  instructor’s  ratings  are  cor¬ 
related  to  the  extent  indicated  by  these  coefficients. 

Of  note  was  tho  strong  tendency  of  tho  Victorville  examining 
board  to  give,  higher  rat  ings  on  tho  final  check  than  on  the  intermedi¬ 
ate  check.  This  result  was  obtained  despito  the  fact  that  examiners 
were  briefed  to  compare  each  student  with  other  students  at  a  similar 
stage  of  training. 

Further  studies  involving  these  ratings  will  be  found  in  chapter  8 
on  the  interrelations  of  proficiency  measures. 


SUMMARY 

The  necessity  of  supplementing  written  tests  of  radar  proficiency 
with  perfonnance  measures  led  Psychological  Research  Project 
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(Radar)  to  construct  six  standardized  performance  checks.  These 
checks  were  for  the  purpose  of  evaluating  student  proficiency  on  air¬ 
borne  radar  missions,  on  supersonic  trainer  missions,  and  on  the  class¬ 
room  radar  equipment.  Each  check  was  constructed  on  the  basis 
of  a  study  of  the  task,  conferences  with  instructional  personnel, 
and  trial  administrations.  A  uniform  format  was  provided  for  all 
checks  and  a  special  arrangement  of  item  pages  with  an  expendable 
scoring  sheet  facilitated  administration.  All  checks  contained  a  state¬ 
ment  of  standard  conditions  under  which  the  check  was  to  bo  adminis¬ 
tered.  Standardized  administration  was  sought  by  the  training  of 
specialized  examiner  boards  at  each  radar  school.  Several  revisions 
of  each  check  were  made  to  provide  measures  applicable  to  different 
radar  equipment  and  different  radar  schools. 

Performance  on  classroom  radar  equipment  was  measured  once,  at 
the  completion  of  bench  set  training.  Proficiency  in  starting,  tuning, 
and  calibrating  the  set  was  evaluated.  Every  step  in  the  operating 
procedure  was  included  in  the  check  and  performance  of  many  steps 
were  required  in  prescribed  sequence.  A  by-product  of  early  check 
dovolopinent  was  the  impetus  given  for  the  development  of  a  standard 
operating  procedure  for  purposes  of  instruction.  The  majority  of 
bench  check  items  were  aimed  at  evaluating  the  procedure  of  adjust¬ 
ing  tho  various  controls  rather  than  the  quality  of  tho  scope  picture 
resulting  from  tho  adjustments. 

Performance  on  the  supersonic  trainer  was  checked  at  three  points. 
Tho  intermediate  navigation  check  evaluated  proficiency  early  in  the 
trainer  course.  Tho  intermediate  bombing  check  measured  proficiency 
in  bombing  after  tho  appropriate  instruction  had  been  given.  The 
final  check  evaluated  performances  on  a  complete  radar  navigation  and 
bombing  mission.  Supersonic  check  items  dealt  both  with  adherence 
to  standard  navigation  and  bombing  procedure  and  with  the  accuracy 
of  the  student’s  results.  Tho  graphic  record  provided  by  tho  trainer 
served  as  a  useful  sourco  of  performance  data.  As  essential  feature 
of  tho  supersonic  checks  was  tho  specific  mission  route  for  each  check; 
this  assured  that  check  items  presented  equivalent  tasks  to  all  students. 

Aerial  performance  was  evaluated  twice.  Tho  intermediate  aerial 
check  measured  proficiency  at  a  point  half  way  through  air-borne 
training.  The  final  aerial  check  was  given  shortly  beforo  graduation. 
Iloth  checks  evaluated  performance  on  typical  aerial  radar  navigation 
and  bombing  missions.  Aerial  check  items,  like  supersonic  trainer 
items,  measured  conformity  to  accepted  methods  of  navigating  and 
bombing  as  well  as  tho  quality  of  the  student’s  results.  Aerial  evalua¬ 
tion  presented  difficulty  in  that  tliero  was  no  permanent  objective  rec¬ 
ord  of  tho  student’s  performance  and  much  depended  on  tho  ability  and 
conscientiousness  of  tho  individual  examiner.  Specific  check  routes 
wero  established  at  each  school. 


124 


So  .2 


Statistical  findings  as  well  as  observation  indicate  that  standardized 
administration  of  the  battery  of  performance  checks  was  only  par¬ 
tially  achieved.  The  practical  demands  of  the  radar  training  program 
did  not  permit  study  of  check-recheck  reliability.  The  one  study  made 
of  a  small  number  of  cases  on  the  final  supersonic  check  suggested 
that  that  check  had  only  moderate  reliability. 

A  study  of  the  relation  between  check  scores  and  instructor  ratings 
on  aerial  missions  resulted  in  substantial  intercorrelations.  The  de- 
'oo  to  which  knowledge  of  check  score  influenced  instructor  judgment 
unknown. 


A  by-product  of  the  radar  project’s  experience  in  constructing  per¬ 
formance  checks  mid  applying  them  to  the  diverse  aspects  of  radar 
observer  training  \vn.-  the  development  of  certain  tentative  generaliza¬ 
tions  about  the  measurement  of  performance.  While  there  was  no 
opportunity  under  wartime  conditions  to  verify  such  generalizations 
experimentally,  they  are  presented  in  this  chapter  because  of  their 
potential  value  to  similar  measurement  efforts  in  the  future. 

The  discussion  will  be  concerned  first  with  questions  related  to  the 
reliability  of  performance  measurement.  As  an  aid  to  this  analysis 
of  reliability,  however,  a  prior  discussion  is  undertaken  of  the  nature 
or  structure  of  performance  check  items.  Here  is  outlined  a,  scheme 
in  terms  of  which  subsequent  discussions  of  reliability  are  oriented. 
Finally,  the  topic  of  validity  is  treated  briefly.  Two  types  of  validity 
are  distinguished  and  the  various  types  of  criteria  against  which  valida¬ 
tion  maybe  carried  out  arc  considered. 

TIIE  STRUCTURE  OF  PERFORMANCE  CHECK  ITEMS 

Rack  ground  for  an  Analysis  of  Sources  of  Unreliability 

As  already  pointed  out  in  chapter  0,  practical  considerations  made 
it  impossible  to  conduct  statistical  studies  of  the  reliability  of  perform¬ 
ance.  checks.  It  was  generally  believed,  however,  that  their  reliability 
was  not  high.  In  order  to  explore  systematically  the  possibilities  for 
improving  reliability,  it  was  concluded  that  some  type  of  analytical 
framework  was  necessary.  The  break-down  of  performance  measure¬ 
ment  which  constitutes  this  framework  is  presented  in  this  section  of 
the  chapter. 

General  Structure  of  Items 

Broadly,  any  performance  check  item  may  be  thought  of  in  terms  of 
two  components — the  student  performance  and  the  examiner  evalua¬ 
tion  of  that  performance.  These  components  are  easily  discerned  in 
n  performance  check  item  in  which  the  examiner  merely  decides 
whether  the  student  behavior  which  he  observes  fits  the  description  of 

— -  f 

1  Written  by  Cpt.  Dougins  W.  Bray. 
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the  correct  behavior  as  given  in  the  item.  Such  items  are  common  in 
the  bench  set  performance  checks  developed  for  the  radar  observer 
program.  For  example,  one  item  asks  whether  or  not  the  st  dent  turns 
a  certain  tuning  knob  counterclockwise.  The  student  performance  in 
this  case  is  the  overt  act  of  turning  the  knob;  the  examiner  evaluation 
of  tlio  performance  consists  merely  of  a  decision  as  to  whether  the 
correct  movement  took  place.  However,  the  two  components  arc  not 
so  obvious  in  other  performance  check  items.  For  example,  an  item 
from  the  supersonic  trainer  performance  check  for  radar  students  asks 
whether  an  inked  track  line  drawn  by  the  trainer  lies  within  5  degrees 
of  where  it  should  lie.  Here  student  behavior  is  not  observed  directly 
as  it  was  in  the  previous  example,  but  is  reflected  through  a  mechanical 
device.  Likewise,  the  examiner’s  evaluation  is  not  a  simple  decision 
but  a  precise  measurement  in  reference  to  an  external  criterion. 

Student  Performance 

A  review  of  performance  cheek  items  reveals  that  the  student  per¬ 
formance  to  be  evaluated  may  be  either  observed  directly  or  inferred 
from  evidence  gained  through  mechanical  devices.  These  two  sources 
of  performance  data  will  be  discussed  separately  below. 

Performance  directly  observed. — Performance  may  be  directly  ob¬ 
served  in  the  student’s  behavior  (action  or  speech)  or  in  his  produc¬ 
tions  (log  entries,  map  plots,  etc.).  Directly  observed  student  action 
is  illustrated  by  an  item  from  the  aerial  performance  check  for  radar 
students  which  asks  whether  the  student  turns  the  receiver  gain  control 
cr  ntcrclockwise.  Directly  observed  student  speech  is  taken.  as  evi- 
d  .jo  of  performance  in  an  item  from  the  same  check  which  asks 
whether  the  student  calls  absolute  altitude  to  the  bombardier  over  the 
interphone.  Student  productions  arc  used  as  performance  data  in  still 
another  item  which  asks  the  time  elapsing  between  certain  entries  in 
the  student’s  log  and  chart. 

Performance  evidenced  through  mechanical  devices. — Performance, 
when  it  is  not  observed  directly,  may  be  evidenced  either  through  the 
equipment  used  by  the  student,  such  as  the  radar  set,  or  by  special 
recording  devices  such  as  the  glass  map  with  marking  pen  attachment 
on  tlio  supersonic  radar  trainer.  The  use  of  operating  equipment — 
in  tli  is  case  the  radar  set — as  a  source  of  performance  data  is  illustrated 
by  tlio  aerial  check  item  which  calls  for  evaluation  of  tho  qunlity'of 
tlio  scope  picture  obtained  by  tho  student.  Here  the  student’s  per¬ 
formance  is  evidenced  not  by  his  behavior  in  turning  the  tuning  con¬ 
trols  but  by  the  effect  of  his  behavior  on  the  radar  scopo  presentation. 
In  another  item,  a  recording  device — tlio  marking  pen  on  the  super¬ 
sonic  trainer — provides  performance  data.  In  this  case  an  inked  track 
line  is  tho  record  evaluated  because  it  represents  the  results  of  the 
student’s  navigational  work. 
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Xu  summary,  t lie  following  arc  the  main  sources  of  performance 
data:  The  student’s  action  or  speech,  the  student’s  productions,  the 
operating  equipment,  and  recording  devices.  The  student  provides 
directly  observed  performance  data,  while  the  equipment  and  record¬ 
ing  devices  provide  indirect  performance  data. 

Evaluation  of  Student  Performance 

The  second  component  of  the  performance  chock  item  is  the  ex¬ 
aminer  evaluation  of  the  student  performance.  The  function  of  the 
examiner,  broadly  defined,  is  to  determine  whether  the  student  per¬ 
formance  satisfies,  within  tolerance  limits,  the  standards  for  accept¬ 
able  performance  set  forth  in  the  check  item.  A  major  difference 
between  items  with  respect  to  their  evaluation,  is  between  those  which 
call  for  evaluation  with  reference  to  a  verbal  description  of  satisfactory 
performance,  and  those  which  specify  that  evaluation  be  made  with 
reference  to  certain  external  criteria.  For  example,  a  rad°r 
check  item  measuring  systematic  habits  of  log  keeping  instructs  the 
examiner  to  give  credit  if  the  student  makes  entries  in  specified  columns 
at  a  given  time.  The  evaluation  is  made  merely  by  coinx^nring  the 
nature  of  the  student’s  log  entries  with  those  called  for  in  the  verbal 
description  within  the  item.  Another  item  from  the  radar  aerial 
check,  on  the  other  hand,  asks  whether  the  student  has  kept  the  air¬ 
craft  within  f>  miles  of  the  briefed  course.  To  evaluate  this  item  the 
examiner  must  first  determine  the  exact  course  of  the  aircraft  from 
t lie  scope  picture  (the  student  performance)  and  then  relate  this  to 
an  external  criterion,  the  briefed  course.  Sinco  wide  divergencies 
exist  between  evaluation  with  reference  to  verbal  description  and 
evaluation  with  reference  to  external  criteria,  the  two  will  be  discussed 
separately. 

Evaluation  in  reference  to  verbal  descriptions. — -When  evaluation 
of  student  performance  is  made  in  reference  to  a  verbal  description, 
tlie  evaluation  consists  of  a  decision  by  the  examiner  who  determines 
whether  the  performance  fits  tho  description  of  the  correct  perform¬ 
ance.  The  conditions  under  which  such  decisions  are  made  may  range 
from  those  so  unambiguous  tha  t  almost  all  examiners  would  evaluate 
a  given  performance  in  the  same  way,  to  those  so  difficult  to  interpret 
that  any  agreement  between  examiners  would  most  probably  be  duo 
to  chance.  At  one  extreme  is  an  item  which  asks  whether  the  student 
turned  a  certain  control  in  a  specified  direction.  It  can  certainly  be 
expected  that  almost  all  examiners  would  score  this  item  in  tho  same 
v-'ay.  However,  an  item  which  asks  whether  the  student  secured  tho 
best  possible  definition  of  scope  returns  within  the  limitations  of  a 
particular  radar  set  involves  a  much  more  difficult  decision,  and  it  is 
probable  that  examiners  wc aid  frequently  disagree. 

Evaluation  in  reference  to  <.  *crnal  criteria. — It  has  been  indicated 
that  the  examiner’s  evaluation  of  performance  is  sometimes  related  not 
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to  a  verbal  description  but  to  external  criteria.  A  consideration  of 
items  of  tbc  latter  type  shows  that  such  criterion  data  are  provided 
primarily  from  two  sources:  the  equipment,  and  recording  devices. 
Foe  example,  in  making  a  bomb  run  with  the  radar  equipment,  a 
“multiple”  drift  correction  is  made  at  the  start  of  the  run.  If  this 
correction  is  made  perfectly,  or  if  subsequent  corrections  within  the 
tolerance  limits  are  made  soon  enough,  the  aircraft  will  pass  directly 
over  the  target.  The  target  constitutes  the  external  criterion  with 
reference  to  which  the  accuracy  of  the  course  correction  is  judged. 
In  an  aerial  performance  check  item,  the  radar  set  serves  as  the  source 
of  information  as  to  the  location  of  the  target.  In  a  supersonic  check, 
evidence  as  to  the  location  of  the  target  comes  from  the  glass  map 
over  which  the  recording  pen  traces  a  line  representing  the  aircraft’s 
course.  Another  illustration  is  found  in  the  item  requiring  that  the 
aircraft  must  be  flown  within  a  specified  distance  of  a  prearranged 
route.  'I'll is  route  serves  as  a  criterion  to  which  the  student’s  actual 
route  is  compared.  In  the  air  the  route  is  found  by  reference  to 
returns  on  the  scope  of  the  radar  set.  Ou  the  supersonic  trainer  it  may 
hi>  easily  determined  by  laying  a  straightedge  on  the  glass  map  between 
the  starting  point  and  the  destination. 

l\xn miner  function x. — From  both  the  discussion  of  the  sources  of 
evidence  regarding  student  performance  and  the  discussion  of  evalu¬ 
ation  in  reference  to  external  criteria,  it  is  clear  that  the  examiner  is 
called  upon  to  perform  a  variety  of  operations.  The  end  result  of 
tlie'-e  operations  is  to  put  either  the  data  reflecting  student  perform¬ 
ance  or  that  representing  the  criterion  in  such  form  that  comparison 
between  them  is  possible.  These  operations  or  examiner  functions  are 
of  three  kinds:  participation,  computation,  and  measurement. 

Some  performance  check  items  require  the  examiner  to  use  the  same 
equipment  and  data  as  the  student  and  to  perform  the  same  task  as  the 
student,  in  order  to  provide  a  standard  against  which  the  student’s 
n  suits  can  be  evaluated.  In  this  discussion  such  examiner  activity  will 
be  called  “participation.”  For  example,  a  method  of  evaluating  the 
accuracy  of  the  student’s  computation  of  the  “multiple”  drift  correction 
in  the  radar  aerial  performance  check  is  to  have  the  examiner  deter¬ 
mine  the  necessary  correction  by  observing  the  scope  simultaneously 
with  the  student.  If  the  heading  correction  which  the  student  inter¬ 
phones  to  tho  pilot  is  within  5  degrees  of  the  heading  correction  found 
necessary  by  tho  examiner,  credit  is  given  on  the  item.  A  complete 
analysis  of  this  item  would  be  ns  follows:  the  evidenco  of  student  per¬ 
formance  is  taken  directly  from  bis  behavior,  i.  e.,  his  speech  when 
interphoning  the  drift  correction  to  the  pilot;  tho  evaluation  by  tho 
examiner  does  not  involve  an  external  criterion,  and  the  examiner 
function  is  participation. 
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in  ilio  supersonic  check  item,  discussed  earlier,  which  evaluated  the 
accuracy  of  the  multiple  drift  correction  in  terms  of  whether  the  air¬ 
craft  passed  over  the  target,  an  additional  task  of  the  examiner  was  to 
note  heading  changes  following  the  initial  correction.  The  algebraic 
.•.uni  of  the  right  and  left  corrections  was  determined  and  evaluated 
in  terms  of  a  tolerance  given  in  the  item.  Such  activity  will  be  called 
‘•coinputiit  ioii‘;  in  the  following  discussion. 

Other  performance  is  evaluated  by  the  use  of  various  measuring  de¬ 
vices.  I  ii  evaluating  the  accuracy  of  a  student's  computation  of  course 
correction  on  a  supersonic  trainer  check,  for  example,  the  examiner 
iiM'--  a  special  protractor.  With  this  he  can  measure  the  angular  dif¬ 
ference  between  the  direction  of  t  lie  inked  track  and  the  true  direction 
to  i he  destination.  This  examiner  function  will  be  called  “measure¬ 
ment.  ”  A  complete  analysis  of  this  item  would  be  as  follows:  student 
performance  is  evidenced  indirectly  through  a  recording  device,  the 
inked  track  line;  evaluation  is  in  terms  of  an  external  criterion  taken 
from  a  recording  device,  the  etched  glass  map  on  which  the  inked  line 
i.-  marked  ;  (he  examiner  function  is  measurement. 

When  the  radar  equipment  provides  the  external  criterion  and  meas¬ 
urement  is  called  for,  the  metrical  devices  intrinsic  to  the  equipment 
are  often  used.  Thus,  in  an  aerial  radar  performance  check,  distance 
..If  course  is  determined  by  using  the.  range  marks  on  the  radar  scope. 
The-e  marks  constitute  a  convenient  distance  scale. 

It  will  he  found  that  in  many  instances  participation  includes  com¬ 
putation  and  measurement.  Many  items  in  the  radar  observer  checks 
call  for  both  computation  and  measurement.  In  some  items  compu¬ 
tation  is  dillicnlt  to  distinguish  from  measurement.  The  three  func¬ 
tions.  however,  are  generally  clearly  di-criminable. 

Di-crclo  ami  Interdependent  Items 

Although  any  performance  check  may  he  analyzed  in  the  foregoing 
manner,  there  are  other  considerations  which  may  considerably  alter 
the  form  of  the  check.  One  important  factor  is  that  the  items  con¬ 
stituting  the  performance  check  may  be  either  discrete  or  interde¬ 
pendent.  If  a  check  is  composed  of  discrete  items,  each  item  places  n 
separate  task  before  the  student.  His  performance  on  an  item  will  not 
he  dependent  upon  his  performance  on  preceding  items.  In  a  per¬ 
formance  cheek  of  flying  ability,  for  example,  the  student  may  he 
fir.-t  cheeked  on  an  Immelman  turn,  then  on  a  loop,  and  so  on.  In 
cheeks  which  are  continuous  or  running  evaluations  of  performance 
in  progress,  however,  the  items  are  interdependent.  A  poor  per¬ 
formance  on  one  item  may  cause  poor  performance  on  another  for 
which  it  is,  in  part,  a  preparation.  For  example,  in  the  aerial  check 
for  radar  observers,  the  student's  computation  of  wind  alTeetS  the 
accuracy  of  his  subsequent  course  correction. 
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All  of  the  performance  checks  developed  to  measure  radar  observer 
proficiency  were  of  t lie  interdependent  type.  It  would  have  been 
theoretically  possible,  of  course,  to  construct  checks  made  up  of  dis¬ 
crete  items.  On  the  supersonic  trainer,  for  example,  a  known  wind,  air 
speed,  and  heading  might  have  been  set  into  the  equipment  and  the 
student  instructed  to  determine  a  “multiple”  drift  correction  on  a  par¬ 
ticular  target.  The  trainer  could  then  have  been  placed  at  another 
point  and  the  student  required  to  take  a  fix.  Here  again  a  separate 
evaluation  could  be  made.  There  were  several  reasons,  however,  for 
tho  use  in  the  radar  program  of  checks  consisting  of  interdependent 
items.  In  the  first  place,  checks  of  performance  in  progress  were 
indicated  because  they  were  more  feasible  to  administer.  The  manipu¬ 
lation  of  a  bomber  in  accordance  with  tho  standardized  conditions  of 
many  separate  tasks  presented  impressive  practical  difficulties.  Sec¬ 
ond,  such  checks  had  the  advantage  of  applicability  to  the  normal, 
training  program  since  check  missions  did  not  have  to  differ  greatly 
from  usual  instructional  missions.  Furthermore,  interdependent 
items  were  much  less  expensive  in  time  since  one  aspect  of  performance 
followed  another  without  interruption  of  performance  or  change  in 
the  testing  conditions.  In  addition,  checks  consisting  of  interde¬ 
pendent  items  were  preferred  by  training  authorities  who  were 
strongly  inclined  towards  evaluating  total  performance  in  progress. 
The  performance  checks  developed  by  tho  Navigator  and  tho  Bom¬ 
bardier  Projects  had  created  a  precedent  for  interdependent  items. 
Finally,  such  an  item  typo  was  suggested  by  the  nature  of  tho  radar 
observer’s  task,  the  components  of  which  arc  themselves  interdepen¬ 
dent.  In  tho  air,  the  student’s  determination  of  such  things  as  wind, 
course  correction,  and  fixes  arc  all  complexly  intertwined.  It  is  quite 
possible  that  a  check  of  the  total  performance  evaluates  the  integration 
of  various  skills  and  so  provides  a  better  estimate  of  complete  pro¬ 
ficiency  than  would  a  separate  evaluation  of  each  skill. 

Although  use  of  interdependent  items  appeared  necessary  in  tho 
radar  program,  thcro  arc,  of  course,  many  situations  in  which  discrete 
items  are  indicated.  In  pilot  training,  for  example,  such  items  wero 
more  in  keeping  with  tho  nature  of  tho  task,  were  easy  to  administer 
in  the  normal  training  program,  ai  d  required  little,  if  any,  extra 
expenditure  of  time.  Where  possible,  of  course,  discrete  items  should 
be  used  since  they  have  a  definite  advantage  over  interdependent  items 
in  terms  of  reliability. 

Direct  and  Indirect  Measurement 

Performance  check  items  may  also  bo  direct  or  indirect  in  their 
evaluation  of  performance.  A  direct  measurement  is  one  in  which 
performance  itself  is  evaluated.  An  indirect  measurement  is  one  in 
which  performance  causally  related  to,  but  distinct  from,  tho  perform- 
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.U1ee  to  lie  evaluated  serves  ns  the  object  of  measurement.  Illustration 
will  make  tli  is  difference  clear:  On  one  of  the  supersonic  cheeks,  the 
accuracy  of  the  student’s  “multiple”  drift  correction  is  judged,  in  part, 
in  terms  of  the  amount  of  subsequent  drift  correction  found  necessary. 
Thus  the  performance  of  the  initial  drift  correction  is  never  checked 
directly  at  all.  Later  behavior  provides  the  data.  Another  some¬ 
what  different  example  is  the  evaluation  of  accuracy  of  wind  deter¬ 
mination  on  an  aerial  mission.  The  only  possible  way  of  evaluating 
this  performance  directly  would  have  been  for  the  examiner  to  com¬ 
pute  a  wind  independently.  Since  this  was  found  to  be  impracticable, 
a  direct  check  of  wind  accuracy  was  not  used.  However,  because 
many  other  aspects  of  aerial  performance,  such  as  course  corrections, 
KTA’s,  and  precision  turns,  which  were  checked  directly,  depend  on  the 
accuracy  of  wind  computation,  a  heavily  weighted  evaluation  of  wind 
computation  was  indirectly  achieved.  It  will  be  clear  that  there  is  a 
close  connection,  on  the  one  hand,  between  discrete  performance  check 
items  and  direct  measurement  and,  on  the  other,  between  interdepen¬ 
dent  performance  check  items  and  indirect  measurement.  In  any 
check  consisting  of  interdependent  items  there  is  the  possibility  of  in¬ 
direct  evaluation — in  fact,  it  is  present  intrinsically  even  if  each  item 
task  is  also  evaluated  directly.  Checks  consisting  of  discrete  item3 
naturally  rest  upon  direct  evaluation. 

RELIABILITY  OF  PERFORMANCE  CHECK  SCORES 

A  variety  of  factors  influence  the  reliability  of  scores  from  per¬ 
formance  measures.  In  the  following  discussion  of  these  factors  they 
are  separated  into  three  broad  categories:  those  which  determine  tho 
reliability  with  which  the  student’s  performance  reflects  his  pro- 
liciency,  those  which  determine  the  reliability  of  the  performance  data 
themselves,  and  those  which  determine  the  reliability  with  which 
student,  performance  is  evaluated.  Discussion  of  the  latter  two  groups 
of  influences  follows  the  analysis  made  in  the  preceding  section. 

Reliability  With  Which  Performance  Reflects  Proficiency 

Since  a  performance  check  is  a  device  for  measuring  student  pro¬ 
ficiency,  it  is  essential  that  proficiency  ho  truly  represented  by  tho 
behavior  measured.  The  sources  of  attenuation  of  performance  ns  an 
index  of  proficiency  may  bo  discussed  under  three  general  headings, 
intrastudent  variability,  variations  in  equipment,  and  variations  in 
testing  conditions. 

Closely  related  to  the  variability  of  performance  as  an  indicator 
of  proficiency  arc  those  variations  which  take  place  from  time  to  tiino 
"it bin  tho  student.  If  tho  student  is  not  motivated  to  do  his  best 
"  oik  on  the  performance  check  or  if  conditions  such  ns  fatiguo  and 
illness  interfere  with  his  performance,  the  score  he  earns  will  not  bo  a 
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good  estimate  of  his  proficiency.  Poor  student  motivation  may  at 
times  he  due  to  the  attitude  of  the  examiner.  Some  examiners  inspire 
a  desire  to  perform  well  while  others  make  the  student  feel  that  the 
check  is  just  one  more  bothersome  requirement. 

In  the  evaluation  of  any  performance  involving  the  use  of  equip¬ 
ment,  it  is  clearly  necessary  that  the  equipment  upon  which  the  task  is 
to  he  performed  must  operate  consistently  from  one  student  to  the 
next.  In  radar,  for  example,  a  student  could  not  he  expected  to 
achieve  good  rcsnlts  if  the  set  were  calibrated  improperly.  It  should 
be  noted,  however,  that  the  importance  of  standard  equipment  is 
related  to  the  kind  of  behavior  to  be  checked.  Check  items  evaluating 
the.  precision  of  results  are  highly  sensitive  to  even  minor  variations 
of  the  equipment.  Items  which  ask  oidy  the  order  in  which  the 
student  performs  certain  operations,  on  the  other  hand,  are  relatively 
immune  to  any  hut  gross  equipment  dillienlties.  In  the  interdependent 
items  found  in  checks  of  performance  in  progress,  however,  even  such 
procedure  items  may  be  alTocted  by  equipment  variation.  For  example, 
one  of  the  radar  aerial  check  items  asks  whether  the  student  recorded 
an  estimated  time  of  arrival  previous  to  a  specified  point.  Poorly 
operating  equipment  might  so  handicap  a  student  that  he  would  have 
no  IOTA  to  record  within  the  time  allowed. 

The  other  major  interference  with  reliable  performance  comes  from 
administering  the  check  under  varying  testing  conditions.  Contribut¬ 
ing  to  variation  in  testing  conditions  arc  such  thing.-,  as  faulty  exami¬ 
ner  behavior,  dilFcrences  in  the  objective  task,  and  inequalities  in 
physical  conditions.  The  examiner  may  be  responsible  for  poor  per¬ 
formance  by  failing  to  acquaint,  the  student  fully  with  the  perform¬ 
ance  expected  of  him  and  the  content  and  conditions  of  the  task. 
Again,  the.  examiner  may  be  at  fault  in  changing  the  objective  difii- 
culty  of  t he  task.  On  the  radar  bench  set,  for  example,  the  examiner 
is  required  to  preset  the  controls  in  prescribed  positions  before  ad¬ 
ministering  the  check.  Failure  to  do  this  makes  the  check  of  a  different 
level  of  difficulty  than  when  conditions  are  established  as  prescribed. 
The  difficulty  of  (lie  task  may  vary,  also,  for  other  reasons.  In  aerial 
radar  performance,  for  instance,  it  is  known  that  terrain  determines 
to  some  extent  the  difficulty  of  nnvigational  and  bombing  problems. 
For  this  reason,  specific  check  mission  routes  were  insisted  upon. 
Finally,  there  is  the  variety  of  physical  conditions  under  which  a 
student  may  >e  required  to  work.  To  again  illnst vote  from  the  aerial 
radar  performance  check,  variations  in  weather,  the  turbulence  of  the 
air,  the  adequacy  and  cooperation  of  the  crew,  the  altitude  and  the 
temperature,  all  may  have  a  considerable  effect  upon  student  perform¬ 
ance.  It  is  obvious  that  every  effort  must  he  made  to  make  the 
objective  task  and  the  environment  in  which  it  is  performed  corn- 
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p;1  ruble  for  nil  students  and  to  assure  equivalent  instruction  of  the 
, mdcnt  by  nil  examiners. 

K(  liability  of  Student  Performance  Dntn 

As  [jointed  out  in  discussing  the  structure  of  performance  check 
items,  student  performance  may  be  observed  directly  in  the  student’s 
behavior  and  productions  or  indirectly  from  the  equipment  of  the 
ta.-k  or  from  recording  devices.  Performance  data  are  most,  subject 
to  error  when  they  are  observed  indirectly.  As  shown  above,  variation 
;tl  the  equipment  and  recording  devices  will  cause  the  student  to 
perform  in  a  manner  which  does  not  truly  represent  his  proficiency. 
However,  even  though  the  student’s  performance  satisfactorily  rc- 
llcets  his  skill,  it  is  still  possible  for  the  equipment  or  recording 
devices  to  present  an  inaccurate  picture  of  his  accomplishment.  In 
administering  the  supersonic  trainer  performance  check,  for  example, 
the  glass  map  on  which  the  student’s  performance  was  recorded  was 
occasionally  found  to  be  out  of  alignment  with  the  artificial  terrain. 
This  resulted  in  a  discrepancy  between  the  actual  position  of  tho 
aircraft,  as  represented  to  the  student  on  the  scope,  and  the  record 
of  that  position  on  the  glass  map.  Under  those  conditions  the  student 
might  have  actually  directed  the  aircraft  exactly  over  the  center  of  a 
town  while  the  record  would  show  that  he  had  missed  the  town 
completely.. 

Another  somewhat  different  possibility  for  error  in  the  data  repre¬ 
senting  student  performance,  has  to  do  with  the  nature  of  the  equip¬ 
ment  or  recording  device  being  used.  Some  instruments  arc  such  that 
accurate  measurement  or  observation  is  possible,  while  others  require 
interpolation,  estimation,  etc. 

Reliability  of  the  Evaluation  of  Student  Performance 

Tn  the  break-down  of  the  structure  of  performance  els  wk  items,  it 
was  seen  that  in  some  eases,  evaluation  of  student  performance  took 
place  with  reference  to  a  verbal  description  within  the  item  while 
in  others  it  was  carried  out  with  reference  to  an  external  criterion. 
In  the  earlier  discussion,  it  was  also  noted  that  evaluations  made  ill 
inference  to  verbal  descriptions  may  vary  from  those  so  simple  that 
all  examiners  may  be  expected  to  agree  to  those  so  difficult  that  any 
agreement  would  probably  he  due  to  chance.  A  review  of  various 
)  "i  fornianee  cheek  items  indicates  that  an  important  factor  in  the 
d'ffnmlty  of  such  evaluations  is  the  degree  of  completeness  of  tho 
dr.-eription  of  the  correct  performance  contained  in  the  check  item. 
This,  in  turn,  is  related  to  the  difficulty  in  objectifying  the  description. 
Tor  example,  an  item  which  asks  merely  whether  or  not  the  student 
turns  n  knob  counterclockwise  contains  a  precisely  formulated  and 
complete  statement  of  the  exported  performance.  On  the  other  hand, 
mi  item  which  asks  whether  a  student  did  something  accurately,  as 
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well  ms  possible,  or  satisfactorily,  leaves  the  decisi on  as  to  the  correct 
performance  largely  to  the  experience  and  subjective  standards  of  the 
examiner.  Evaluations  in  reference  to  verbal  descriptions  can  be 
made  more  reliable,  in  part,  by  defining  acceptable  performance  as 
precisely  and  completely  as  possible,  in  part,  by  selecting  as  examiners 
highly  experienced  personnel,  and,  in  part,  by  training  examiners 
thoroughly  in  an  attempt  to  standardize  their  concept  of  the  correct 
performance  for  the  appropriate  items. 

When  evaluation  takes  place  in  reference  to  an  external  criterion, 
the  question  of  reliability  hinges  in  part  upon  the  nature  of  the 
criterion  utilized.  If,  as  in  the  case  of  a  target  or  a  route  on  the  glass 
map  of  the  supersonic  trainer,  the  criterion  maybe  precisely  identified, 
then  measurement  in  relation  to  it  may  bo  expected  to  be  reliable.  If, 
on  the  other  band,  the  criterion  can  not  be  precisely  established,  some 
unreliability  of  measurement  must  result 

From  tl ic  point  of  view  of  the  examiner  functions  of  participation, 
computation,  and  measurement,  as  described  earlier,  reliability  is 
influenced  in  several  ways.  Participation  appears  most  likely  to  re¬ 
sult  in  unreliability  since  it  assumes  a  high  degree  of  examiner  care 
and  skill  in  the  performance  being  checked.  The  reliability  of  items 
relying  on  computation  may  be  improved  by  constructing  such  items 
so  that  the  numerical  operations  involved  are  the  simplest  possible. 

The  function  of  measurement  may  be  made  more  dependable  by  pro¬ 
viding  metrical  devices  to  facilitate  the  examiner’s  task. 

The  Measurement  of  Reliability 

Measurement  of  the  reliability  of  the  performance  check  scores  pre¬ 
sents  several  difficulties  of  definition  and  methodology.  In  this  dis¬ 
cussion,  tiro  coefficient  of  reliability  will  be  understood  to  represent  the 
extent  of  agreement  between  scores  which  students  receive  on  a  per¬ 
formance  check  taken  under  routine  administrative  conditions  and 
scores  which  they  would  have  received  had  they  taken  the  cl  ack  at  ft 
different  time  with  exnm’uicr,  equipment,  recording  device,  and  con¬ 
ditions  of  administration,  varied  as  1 1  toy  are  under  routine  adminis¬ 
trative  conditions.  An  important  provision  in  this  definition  is  that 
the  reliability  of  (lie  performance  check  score  is  not  independent  either  ; 
of  intrnstndeut  variability  or  of  variations  in  examiner,  equipment, 
etc.  In  the  routine  use  of  performance  checks,  such  independence  is 
not  found.  Conventional  methods  of  determining  reliability  are  dis¬ 
cussed  below  in  terms  of  this  definition.  i  I 

Thcro  nro  t  wo  principal  internal  methods  of  computing  reliability:  ( 
t ho  part-part  or  split-lmlf  method,  and  the  odd-even  method.  The  r 
part-part  method  correlates  the  score  earned  on  one  half  of  the  check 
with  that  earned  on  tho  other  half,  while  the  odd-even, method  corre-  . 
lates  tho  score  earned  on  odd  items  with  that  earned  on  even  items. 
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W*ii In of  those  methods  is  suitable  for  determining  the  reliability  of 
;1  performance  check  since  they  roly  on  :i  single  administration  of  the 
lucasure.  Within  a  single  administration,  several  important  sources 
uf  variation  between  administrations  are  held  relatively  constant, 
'lin.-e  arc  intrastudent  variability  in  performance,  variation  between 
examiners,  variation  in  testing  conditions,  variation  in  testing  equip¬ 
ment,  etc. 

An  alternate  method  of  determining  performance  check  reliability 
is  by  the  test-retest  method.  This  may  be  accomplished  either  by  n 
second  administration  of  the  same  check  or  by  using  an  uUcrnuto  form 
of  the  measure  for  the  reehcck.  When  cither  of  these  methods  is  used, 
learning  and  memory  will  act  to  increase  scores  on  the  recheek  over 
the  check.  This,  of  course,  would  not  at  tenunto  tho  coedicient  of  relia¬ 
bility  if  it  were  not  for  the  fact  that  amount  of  improvement  will 
vary  from  student  to  student.  A  point  of  note  in  connection  with  uso 
of  tho  test-retest  method  is  that  no  attempt  should  bo  mndo  to  stand¬ 
ardize  student  motivation  and  training,  examiner,  equipment,  re¬ 
cording  devices,  or  conditions  of  administration  more  completely  than 
would  be  the  ense  in  the  normal  testing  routine. 

The  construction  of  an  equivalent  test  form  presents  numerous  prac¬ 
tical  difficulties  in  the  case  of  checks  of  performance  in  progress  con¬ 
sisting  of  interdependent  items.  In  such  instances,  all  critical,  mens¬ 
urable  aspects  of  the  task  are  presumably  included  in  tho  original 
form.  While  it  may  not  he  possible  to  create  a  second  check  of  alter¬ 
nate  equivalent  tasks,  it  is  in  some  instances  possible  to  vary  the  content 
of  the  task.  In  aerial  radar  performance  checks,  for  example,  chang¬ 
ing  the  route  of  tho  mission  would  result  in  a  rcchcck  similar  to,  hut 
not  equivalent  to,  the  original  check. 

The  coedicient  of  reliability  described  above  expresses  the  reliability 
of  the  performance  cheek  under  routine  administration.  It  is  occa¬ 
sionally  useful  to  discover  just  how  reliable  the  check  is  when  admin¬ 
istered  by  different  examiners  but  with  all  other  conditions  held  con¬ 
stant.  To  distinguish  this  concept  of  reliability  from  the  luoro 
general  one,  it  will  be  referred  to  as  the  coellicient  of  objectivity.  Tho 
term  objectivity  was  chosen  because  differences  between  evaluations 
by  different  examiners  arc  presumably  duo  to  subjective  differences  in 
their  use  of  the  performance  check. 

To  d '(ermine  the  coedicient  of  objectivity  of  a  performance  check, 
it  is  necessary  to  eliminate  extra-examiner  variability.  In  many  situ¬ 
ations  tho  only  practicable  way  to  do  this  is  to  have  two  examiners 
u  c  the  check  independently  to  evaluate  the  sumo  performance.’  A 
correlation  between  these  two  sets  of  figures  will  produce  the  desired 
coedicient. 


1  Tl-.li  tc(  linlqiio  wan  lined  by  IViyrlioJocli-iU  UrMMirch  Project  (Pilot)  nnd  waa  called 
"observer  reliability.” 
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In  1 1 10  <awe  \vlu>rc  (he  two  sets  of  examiners  agree  perfectly,  t ! ie 
coeHieicnt  of  objectivity  will  be  1.00  and  a  lower  coellicient  of  relia¬ 
bility  will  mean  that  all  unreliability  comes  from  variation  in  testing 
conditions,  student  perfoi niance,  equipment,  recording  devices,  etc. 
In  the  case  where  the  coellicient s  of  objectivity  and  reliability  an* 
equal,  it  will  mean  that  such  variables  do  not  influence  check  score* 
and  that  all  unreliability  is  due  to  lack  of  examiner  agreement  (lack 
of  objectivity  of  the  measure).  Such  information  reveals  whether 
eirort.  expended  to  produce  .standardized  performance  checking  should 
he  directed  toward  improving  the  instrument  and  its  administration 
or  toward  eliminatin';  extra-cheek  variability.  When  it  is  found  (hat 
the  coefficient  of  objectivity  is  too  low  to  he  acceptable,  tet rachoric 
correlations  for  the  scoring  of  each  item  l>y  the  two  sets  of  examiners 
will  point  out  those  items  which  are  most  lacking  in  objectivity. 

VALIDITY  OF  THE  PERFORMANCE  CHECK 

In  reference  to  proficiency  measures,  two  different  concepts  of  valid¬ 
ity  limy  he  distinguished.  According  to  the  usual  concept,  a  measure 
is  valid  if  scores  from  it  predict  future  attainment.  For  convenience 
of  discus, siou  this  will  he  referred  to  as  predictive  validity.  In  radar 
observer  basic  training,  for  example,  the  predictive  validity  of  per¬ 
formance  checks  would  he  determined  from  the  lelationship  between 
performance  check  scores  and  one  of  several  other  criteria  of  profi¬ 
ciency.  These  criteria  might  represent  proficiency  in  combat  or  in  ad¬ 
vanced  training,  or  they  might  even  lie  other  measures  from  basic 
training.  The  second  concept  of  validity  holds  a  measure  to  lie  valid 
if  it  furnishes  a  comprehensive  test  of  achievement  for  a  gi\en  area  of 
instructional  material.  Validity  of  this  type  will  be  referred  to  as 
curriculum  validity.  The  curriculum  validity  of  performance  checks 
would  be  judged  in  terms  of  the  extent  to  which  they  tested  the  skills 
taught  in  the  sections  of  the  training  course  they  were  intended  to 
cover. 

The  degree  of  relationship  between  predictive  validity  and  curric¬ 
ulum  validity  may,  of  course,  he  of  any  magnitude.  If,  for  example, 
a  course  of  i list  rnet  ion  is  poor  preparat  ion  for  future  proficiency,  meas¬ 
ures  possessing  high  curriculum  validity  may  he  poor  predie.tois  of 
later  performance.  On  the  other  hand,  if  proficiency  in  training  is 
highly  related  to  later  proficiency,  measures  having  high  curriculum 
validity  will  have  high  predictive  validity.  The  performance  checks 
developed  for  radar  observer  training  were  pointed  toward  curriculum 
validity,  since  their  purpose  was  to  provide  a  complete  evaluation  of 
student  proficiency  during  the  basic  training  course.  It  would  have 
been  highly  desirable,  of  course,  to  have  been  able  to  determine  their 
predictive  validity. 
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Curriculum  Validity 

Tin*  initial  work  on  n  proficiency  measure  nnird  he  pointed  toward 
nii  rit  nliim  validit}’.  Even  if  suitable  criteria  of  predictive  validity 
:lll.  obtainable,  routine  use  of  the  measures  can  seldom  be  postponed 
uni  il  complete  validation  studies  can  be  made.  The  use  of  curriculum 
validity  is,  however,  not  wholl}'  based  on  negative  reasoning.  Curric¬ 
ulum  validity  is  usually  achieved  in  cooperation  with  training  au¬ 
thorities.  From  the  point  of  view  of  the  practical  training  situation, 
there  can  be  little  quarrel  with  the  validity  of  a  performance  check  if 
it  measures  those  abilities  in  which  the  student  must  he  proficient  bo- 
fore  he  is  considered  ready  to  progress  to  a  later  stage  of  training. 
The  most  effective  use  of  expert  opinion  in  establishing  curriculum 
validity  is  achieved  by  obtaining  expert  criticism  of  proficicny  meas¬ 
ures  developed  by  psychologically  trained  personnel.  This  method 
is  facilitated  if  such  psychological  personnel  have  achieved  modernto 
prolicienc}-  in  the  performance  to  he  evaluated. 

Predictive  Validity 

Problems  associated  with  the  quest  ion  of  the  predictive  validity  of  a 
proficiency  measure  relate  primarily  to  the  criteria  against  which 
validation  should  be  carried  out.  These  criteria  fall  into  several  cate¬ 
gories:  measures  of  final  proficiency,  measures  of  proficiency  in  later 
training,  and  alternate  measures  of  proficiency  within  the  same  stage 
of  training. 

Final  proficiency  as  a  criterion. — Had  it  been  practicable,  it  would 
have  been  of  considerable  value  to  determine  for  proficiency  measures 
their  predictive  validity  against  a  combat  criterion.  However,  tho 
difficulties  of  collecting  precise  prolicienc}'  duty  on  radar  observers 
engaged  in  aerial  combat  were  great  and  such  efforts  as  were  mado 
bore  little  fruit.  The  value  of  criteria  which  might  have  been  ob¬ 
tained,  such  as  bombing  accuracy,  would  have  been  seriously  attenu¬ 
ated  by  such  variable  influences  as  type  aircraft,  effectiveness  of  crew, 
and  equipment. 

Proficiency  in  later  training  as  a  criterion. — Indices  of  subsequent 
proficiency  may  also  ho  obtained  from  measures  of  ability  in  advanced 
stage:-,  of  t  raining.  In  the  preparation  for  radar  observers  for  combat 
a  second  period  of  training  followed  graduation  from  the  basic,  train¬ 
ing  schools.  'This  training  took  place  after  the  combat  crew  had 
been  assembled  and  was  intended  to  organize  the  several  members  of 
she  crew  into  nn  efficient  team.  Ilad  a  suitable  measure  of  proficiency 
at  this  level  been  available,  it  would  have  provided  a  useful  criterion 
for  the  validation  of  proficiency  measures  in  basic  training.  How¬ 
ever,  experience  with  such  data  show  them  to  have  serious  weaknesses. 
The  di flic u lties  of  securing  reliable  measurement  of  individual  profi¬ 
ciency  in  crew  training  are  at  least  as  great  as  they  are  at  the  individual 
training  level.  To  the  variable  influences  of  equipment  and  of  the 
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physical  conditions  of  aerial  performance  is  added  (he  systematic 
error  of  measuring  individual  performance  in  a  team  activity.  The 
performance  of  the  radar  observer  is  a  function  not  only  of  his  own 
proficiency  but  also  of  that  of  the  other  crew  members. 

Unless  there  is  assurance  that  proficiency  measures  from  a  subse¬ 
quent  stage  of  training  are  free  of  such  faults,  they  are  of  little  practi¬ 
cal  value  as  criteria.  The  development  of  such  assurance  frequently 
necessitates  ambitious  research  at  the  advanced  training  level.  More¬ 
over,  there  is  almost  invariably  a  long  waiting  period  before  such 
criterion  data  mature.  In  the  pr  ctical  situation  it  may  be  impossible 
to  postpone  the  administrative  i.  of  proficiency  measures  while  await¬ 
ing  the  results  of  future  measurement  of  proficiency. 

MEASURES  OI<  PROFICIENCY  W  ITHIN  THE  SAME  STAGE  OF 

TRAINING  AS  CRITERIA 

1.  /Scores  made  by  good  and  poor  students  as  a  criterion. — A  meas¬ 
ure  of  the  predictive  validity  of  the  performance  chock  at  the  training 
level  may  be  obtained  by  administering  the  check  to  students  of  differ¬ 
ent  proficiency.8  The  students,  in  such  a  case,  either  may  be  determined 
by  some  independent  means  to  be  of  different  proficiency  or  may  be  as¬ 
sumed  to  differ  in  skill  by  reason  of  having  undergone  different 
amounts  of  training.  In  the  former  instance,  the  independent  evalu¬ 
ation  of  proficiency  will  have  been  based  on  previous  performance 
measures  such  as  instructor  ratings.  The  use  of  extremely  good  and 
extremely  poor  groups  of  students  increases  the  likelihood  of  getting 
two  groups  that  arc  actually  different  in  proficiency. 

Essentially  similar  to  the  use  of  check  scores  made  by  students  of 
varying  proficiency  is  the  use  of  the  graduation-elimination  criterion 
for  validating  the  performance  check.  It  is  important,  of  course,  that 
tho  check  score  being  validated  not  enter  into  the  determination  of 
graduation-elimination,  since  this,  of  itself,  would  produce  a  sizeable 
correlation.  Wlicro  the  grading  system  is  not  acceptably  reliable,  the 
uso  of  graduation-elimination  is  of  little  value  as  a  criterion.  For 
practical  usefulness,  it  is  also  necessary  that  a  significant  number  of 
eliminations  bo  made.  In  the  radar  program,  for  example,  the  need 
for  radar  observers  did  not  permit  the  elimination  of  more  than  a  few 
students,  and  graduation-elimination  was  therefore  of  little  value  as 
n  criterion. 

2.  End  result  of  performance  as  a  criterion. — Since  it  is  frequently 
impracticable  to  uso  measures  of  subsequent  proficiency  as  criteria 
against  which  to  determine  validity,  more  readily  available  sources 
of  criterion  data  should  be  considered.  One  such  source  is  tho  measure¬ 
ment  of  the  end  result  of  tho  performance  being  measured.  In  radar 

•  Sucli  a  technique  ivns  used  In  Psychological  Heeenrch  Project  (Pilot). 
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0|  ,.rviT  training  (lie  primary  objective  of  the  student’s  performance 
u  is  to  bond)  a  target.  1 1  is  circular  error,  the  distance  away  from 
.i,e  target  that,  the  bomb  actually  hit,  would  appear  to  bo  n  valuable 
criterion.  However,  to  be  of  value,  a  criterion  must  not  only  be  an 
obvious  consequence  of  the  performer's  skill;  it  must  also  be  reliable. 
'The  usefulness  of  circular  error  was  expected  to  be  limited  because  of 
the  previously  reported  findings  of  the  Bombardier  Project  concerning 
die  influence  of  variables  other  than  student  ability  on  bombing  accu¬ 
racy.  These  variables  included  equipment,  the  nircraft,  llio  weather, 
and  other  crew  members.  In  addition,  it  must  be  remembered  tlmt 
the  circular  error  resulting  from  any  navigation  lcg-and-bomb  run  is 
only  a  single  measure  of  proficiency.  It  is  known  that  the  larger  tho 
number  of  measurements  obtained,  the  more  reliable  tends  to  bo  tho 
evaluation  of  performance.  The  gathering  of  data  for  many  bomb 
drops  would  presumably  have  offset  variable  effects  but  the  number 
of  drops  needed  was  expected  to  greatly  exceed  the  number  possible 
in  tho  regular  training  program.  The  use  of  the  end  results  of  per¬ 
formance  as  validation  data  is  dependent  upon  tho  possibility  of 
making  such  results  meet  standards  of  satisfactory  reliability.1 

h.  / ntercorreiations  of  proficiency  measures  as  a  criterion. — Tho 
validity  of  performance  measures  has  occasionally  been  inferred  from 
their  intercorrelations.  It  is  clear,  however,  that  if  none  of  the  pro¬ 
ficiency  measures  which  are  intcrcorrclated  is  known  to  have  predictive 
validity,  their  intercorrelations,  of  themselves,  can  tell  us  nothing  of 
predictive  validity.  Furthermore,  even  if  one  of  the  measures  is  of 
known  predictive  validity,  its  correlation  with  other  measures  may  tell 
little  of  their  validity  because  the  other  measures  may  correlate  merely 
with  that  portion  of  the  valid  measure  which  is  not  predictive  of  tho 
criterion. 

Neither  is  this  method  particularly  helpful  in  determining  the  cur¬ 
riculum  validity  of  performance  checks.  Even  if  one  check  is  known 
to  he  a  complete  and  reliable  measure  of  achievement  in  the  course  of 
training,  the  fact  that  another  check  of  a  specific  section  of  the  course 
does  not  correlate  highly  with  it  does  not  indicate  conclusively  that 
the  latter  check  is  invalid  as  a  measure  of  the  task  it  evaluates.  It 
may  well  be  that  the  more  specific  check  is  devoted  to  evaluating  tho 
Earning  of  a  task  in  which  individual  differences  disappear  through 
overlearning  by  the  time  the  measure  of  final  proficiency  is  taken. 
For  example,  a  check  of  achievement  on  the  bench  set  might  bo  a 
fdisfnctory  measure  of  achievement  in  that  specific  training  situation 
find  yet  correlate  poorly  with  the  final  aerial  check  because  early 
differences  in  proficiency  in  elementary  set  operation  arc  a  minor  factor 
m  later  radar  navigation  and  bombing. 

*  l'or  a  report  on  the  reliability  of  circular  error  la  rndar  bombing  nnJ  Itn  correlation 
'Ci'U!i  performance  chceku,  sec  cli.  9. 
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SUMMARY 

Experience  with  the  construction  and  use  of  performance  checks  in 
the  radar  observer-training  program  is  summarized  in  this  chapter, 
Suggestions  are  made  regarding  possible  sources  of  unreliability  tJf 
measurement,  methods  of  determining  reliability,  and  the  evaluation 
of  validity. 

As  an  aid  to  the  discussion  of  reliability,  the  performance  check  is 
analyzed  from  the  point  of  view  that  it  comprises  two  primary  com¬ 
ponents  :  student  performance,  and  the  evaluation  of  that  performance 
by  the  examiner.  In  a  further  analysis  it  is  noted  that  student  per¬ 
formance  may  be  observed  directly  in  student  behavior  and  in  student 
productions  and  indirectly  through  operating  equipment  and  recording 
devices.  Evaluation  of  student  peformance  by  the  examiner  may  be 
made  either  in  reference  to  a  verbal  description  of  satisfactory  per¬ 
formance  or  in  reference  to  external  criteria.  In  radar  observer  per- 
iorniance  checks  there  are  two  sources  of  external  criteria  :  the  operat¬ 
ing  equipment  and  recording  devices.  In  the  course  of  his  evaluation 
the  examiner  is  called  upon  to  perform  a  variety  of  operations,  the 
end  result  of  which  is  to  put  either  the  data  reflecting  student  per¬ 
formance  or  that  representing  the  criterion  in  such  form  that  com¬ 
pulsion  between  them  is  possible.  These  operations  or  examiner 
functions  arc  of  three  kinds:  participation,  computation,  and  measure¬ 
ment..  In  systematically  exploring  the  performance  chock  from  the 
point  of  view  of  possible  sources  of  unreliability,  the  above  analysis  is 
followed. 

In  a  discussion  of  the  measurement  of  performance  check  reliability, 
it  is  pointed  out  that  the  test-retest  method  provides  the  only  entirely 
suitable  results.  Internal  measures  of  reliability  neglect  important 
sources  of  variation  between  successive  check  administrations.  As  a 
rough  guide  to  the  sources  of  unreliability,  a  coefficient  of  objectivity 
is  distinguished  from  the  coefficient  of  reliability.  The  former  coef¬ 
ficient  describes  the  reliability  of  the  cheek  when  administered  by 
different  examiners  but  with  all  other  conditions  bold  constant. 

When  this  coefficient  is  low,  it  is  taken  to  mean  that  the  check  is 
insufficiently  objective  for  consistent  application  by  two  examiners. 
This  suggests  attention  to  the  mechanics  of  the  check  rather  than  to 
increased  standardization  of  testing  conditions,  equipment,  or  other 
va  riables. 

Two  concepts  of  validity  arc  distinguished.  A  measure  is  said  to 
have  predictive  validity  if  scores  from  it  predict  future  attainment. 
It  is  said  to  have  curriculum  validity  if  judged  by  experts  to  test 
adequately  the  skills  taught  in  that  section  of  the  training  course  it  is 
intended  to  cover.  Criteria  against  which  predictive  validity  may  be 
determined  fall  into  three  categories:  measures  of  final  or  combat 
proficiency,  measures  of  proficiency  in  later  stages  of  training,  and 
alternate  measures  of  proficiency  within  the  same  stage  of  training. 
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CHAPTER  EIGHT _ ___ 

Interrelations  of  Proficiency 

Measures 1 


In  this  chapter  the  interrelations  of  the  printed  tests  and  per¬ 
formance  chocks  discussed  in  preceding  chapters  are  examined.  There 
arc  generally  considered  to  he  four  basic  radar  observer  skills,  namely, 
radar  navigation,  radar  bombing,  set  operation  and  scope  interpreta¬ 
tion.  These  skills  are  not  discrete;  there  is  considerable  overlapping 
and  interaction  among  ihem.  For  each  skill  except  scope,  interpreta¬ 
tion,  printed  tests  and  standardized  performance  checks  have  been 
developed.  These  tests  and  checks  constituted  a  battery  of  11  profi¬ 
ciency  measures  administered  routinely  at  several  radar  observer 
schools. 

As  a  basis  for  analyzing  the  interrelations  of  these  measures,  their 
intercorrelations  were  determined.  The  coefficients  obtained  will  bo 
interpreted  from  several  points  of  view.  First  examined  aro  the 
correlations  between  tests  and  performance  cheeks  which  measure 
similar  skills.  The  question  is  raised  as  to  whether  or  not  such  correla¬ 
tions  arc  high  enough  to  warrant  the  substitution  of  administratively 
economical  printed  tests  for  administratively  expensive  performance 
checks. 

Next  considered  are  certain  questions  in  connection  with  the  corre¬ 
lation  between  pairs  of  performance  checks  and  pairs  of  printed  testa. 
Among  these  are  the  degree  of  relationships  between  measures  of  the 
same  skill  as  compared  to  mcasmes  of  different,  skills,  and  the  magni¬ 
tude  of  the  intereorrelations  yielded  by  different  measures  of  the  same 
basic  skill.  Where  these  correlations  involve  performance  cheeks, 
they  are  compared  to  parallel  correlations  between  instructor  ratings 
of  proficiency. 

Correlations  between  measures  of  one  skill  with  thee  of  another 
arc  next  examined.  Evidence  is  sought  as  to  the  magnitude  of  rela¬ 
tionship  between  proficiency  in  different  skills  and  tho  question  is 
raised  as  to  whether  certain  skills  are  more  highly  correlated  than 
others. 

’Written  by  S^t.  Philip  II.  Krledt  with  tin*  naslatance  of  R/Sgt.  Iiolnnd  K.  Jobnaton  and 
Ilnrold  I*\  Kunsnmn.  . 
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Finally,  where  correlations  ure  available  for  parallel  measure 
used  elsewhere  in  the  AAF  Aviation  Psychology  Program,  those  tu0 
introduced  for  comparative  purposes. 

Till*  VARIABLES  AND  THE  SAMPLE 

The  correlation  coefficients  reported  in  this  chapter  are,  with  a 
few  exceptions,  based  on  the  scores  of  190  students  who  were  gradu- 
ated  from  Langley  Field.  The  students  were  members  of  classes 
45-19  through  45-33.  All  were  rated  bombardiers  with  no  combat 
experience.  Class  45-19  graduated  on  12  May  1945;  class  45-33,  on 
IS  August  1945. 

This  sample  was  selected  in  order  to  host  meet  the  following  require¬ 
ments: 

1.  A  group  of  students  was  desired  who  had  taken  all  of  the 
important  proficiency  measures.  This  would  make  it  possible  to  base 
all  correlations  on  the  same  sample  and  thus  render  comparisons  of 
correlation  coefficients  more  meaningful.  For  most  of  the  correla¬ 
tions,  the  N  is  190  or  very  close  to  it.  In  a  few  instances,  however,  the 
N  drops  to  as  low  as  1G0.  Because  of  the  emergency  need  to  graduate 
as  many  radar  observers  as  possible,  students  occasionally  missed 
taking  some  of  the  proficiency  measures.  For  those  tests  and  checks 
considered  most  important,  however,  the  N  is  over  185. 

2.  Students  were  desired  who  had  taken  the  performance  checks 
during  a  period  when  they  were  believed  to  be  most  reliably  ad¬ 
ministered.  In  choosing  this  period  it  was  assumed  that  scores  given 
by  relatively  experienced  examiners  would  be  more  reliable  than 
scores  given  by  the  less  experienced  examiners  who  administered  pro¬ 
ficiency  measures  early  in  the  training  program. 

3.  Students  were  desired  from  a  period  during  which  the  fewest 
changes  were  made  in  the  forms  of  the  proficiency  measures.  For 
Iho  sample  selected,  each  printed  test  was  administered  in  only  ono 
form;  however,  of  the  six  performance  checks,  four  were  given  in 
two  slightly  different  forms. 

Variables 

The  fivo  printed  tests  referred  to  in  this  chapter  are  described 
briefly  below.  The  development  of  these  tests  is  described  in 
chapter  5. 

Final  Text  /  for  AN/APS-15,  AN/APS-15A ,  and  AN/AFQ-1S , 
form  B  ( Pla-B ). — This  test  is  divided  into  four  sections.  Section  A 
is  designed  to  measure  scope  interpretation  and  navigation  skills 
through  the  technique  of  “flying”  a  simulated  mission.  Students  arc 
provided  with  an  expendable  Mercator  map  of  the  area  in  which  tho 
navigation  and  bombing  is  to  tnko  place,  10  scope  photographs  at 
various  points  on  the  route,  and  readings  which  would  bo  obtained 
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j'l  >u  various  navigational  instruments.  With  this  information  the 
utnclcnt  uses  navigational  tools  to  take  the  computations  required  at 
various  points.  Section  B  is  designed  to  measure  skill  in  using  the 
[-XiB  computer  to  solve  ratlnr  navigation  problems.  Section  C  is 
designed  to  measure  skill  in  determining  the  position  of  the  aircraft 
fi'om  simulated  scope  photographs  and  in  plotting  this  position  on 
a  map.  Section  D  is  designed  to  measure  skill  in  the  navigational 
.technique  of  wind  determination  by  airplot.  Throughout  the  chapter 
this  test  is  referred  to  ns  Final  Test  I. 

Final  Tent  II  for  AN / APS- 15,  form  B  ( Plb-B ). — This  test  is  di¬ 
vided  into  two  sections.  Section  A  contains  items  testing  technical 
information  about  radar  bombing  and  radar  navigation  procedures. 
Emphasis  is  placed  on  radar  bombing.  Section  B  contains  items  on 
technical  aspects  of  tuning,  use  of  the  beacon,  and  other  phases  of  set 
operation.  This  test  was  used  with  Final  Test  I  as  a  comprehensive 
examination.  Throughout  this  chapter  it  is  referred  to  as  Final 
Test  II. 

Radar  bombing  intermediate  test  foi'  AN /APS-16,  form  B  ( Pie- 
B). — This  test  contains  items  on  the  theory  and  technique  of  bombing, 
including  the  use  of  equipment  controls  before  and  during  the  bomb 
1  un.  Throughout  this  chapter  it  will  be  referred  to  as  the  radar  bomb¬ 
ing  intermediate  test. 

Radar  navigation  intennediate  test  for  AN/APS-15,  form  B  ( Plhr~ 
B). — This  test  is  divided  into  three  parts.  Part  I  consists  cf  items 
dealing  with  the  operation  of  radar  equipment  and  the  techniques 
involved  in  navigating  from  radar  returns.  Part  II  consists  of  prob¬ 
lems  dealing  with  the  use  of  the  E-6B  computer  in  basic  and  radar 
navigation.  Part  III  contains  problems  in  the  determination  of  wind 
by  air  plot.  Throughout  the  chapter  this  test  will  be  referred  to  as  the 
radar  navigation  intermediate  test. 

Set-  operation  intermediate  test-  for  AN/  APS-15,  form  A  ( Plk-A ) 
end  AN/APS-15A,  form  A  ( Pll-A ). — This  test  is  divided  into  three 
sections.  Section  A  deals  with  the  operation  of  AN/APS-15  equip¬ 
ment  in  searching  for  targets  and  beacons;  with  the  location,  function, 
and  effects  of  malfunction  of  various  components  of  the  equipment; 
and  with  the  operation  of  the  :  <>t  in  bombing.  Section  B  deals  with  the 
SCR-718A  altimeter,  a  radar  device  used  to  determine  absolnto  alti¬ 
tude,  and  with  the  adjustment  of  the  AN/APS-15  sot  in  terms  of  read¬ 
ings  from  the  altimeter.  Section  C  differs  from  sections  A  nnd  B  in 
that,  it  covers  material  specific  to  the  AN/APS-15  A  set.  Throughout 
the  chapter  this  test  is  referred  to  as  the  set  operation  intermediate  test. 

The  development  of  the  performance  checks  to  bo  discussed  in  this 
chapter  is  described  in  chapter  G.  Brief  descriptions  of  these  ch.ccka 
are  given  below. 

Pinal  benrh  set  performance  check  for  AN / APS-lfi,  form  B  (PCa~ 
B). — Each  item  in  this  check  defines  concisely  a  distinct  step  in  the 
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procedure  used  in  (lie  preoperational  cheek,  in  sturtin^  ;md  tuning  j„ 
range  :tiul  altitude  calibration,  and  in  the  turn-off.  The  student  is  told 
by  the  examiner  to  go  through  a  given  procedure  and  call  out  each  step 
us  it  is  performed.  If  an  item  is  omitted  or  performed  incorrectly,  the 
examiner  corrects  the  student  immediately.  Throughout  the  chapter 
this  check  is  referred  to  as  the  final  bench  set  check. 

Final  supersonic  trainer  performance  check  for  AN/APS-lh, 
AN/APS-l~>A,undAN/APQl3,formA  ( P7a-A )  and  Langley  form 
B  ( LP'/u-B ). — Two  forms  of  this  check  were  administered  to  different 
students  in  the  sample  studied.  The  forms,  are  essentially  the  same; 
each  is  a  combination  of  the  intermediate  checks  for  navigation  and 
bombing  described  below.  Where,  the  scores  available  were  for  form 
B,  such  scores  were  transformed  to  scores  comparable  to  those  for 
form  A  (P7a-A).  Throughout  the  chapter  these  checks  are  referred 
to  as  the  final  supersonic  check. 

Radar  bombing  supersonic  intermediate  cheek  for  A N / A, PS-15, 
AN / APB-15A,  and  A N / APQ-13,  fernn  A  ( P7c~A ),  and  Langley 
form  B  (L  INc-B) . — Two  forms  of  this  check  arc  listed  for  the  same 
reason  as  stated  below.  Scores  from  form  B  were  transformed  to 
scores  comparable  to  those  for  form  A.  Form  A  contains  items  cov¬ 
ering  both  direct  and  coordinated  bombing,  while  Langley  form  B 
tests  coordinated  bombing  only.  Differences  between  the  two  forms 
are  thought  to  bo  minor.  Throughout  this  chapter  these  checks  are 
referred  to  as  the  bombing  supersonic  intermediate  check. 

Radar  navigation  supersonic  intermediate  check  for  AN /APB-15, 
AN/ APS-lnA,  and  AN/ APQ-13,  form  CX,  and  Langley  form  C 
( L  P/e-C). — Both  forms  of  this  cheek  are  designed  to  measure  skill 
in  the  navigation  procedures  required  to  conduct  a  simulated  radar 
navigation  mission  successfully  on  the  supersonic  trainer.  The  scores 
on  form  C  were  transformed  to  scores  comparable  to  those  for  form 
OX.  Throughout  this  chapter  these  checks  arc  referred  to  as  the 
navigation  supersonic  intermediate  check. 

Final  aerial  performance  check  for  AN /APS-lb,  A  N/A  PS-15  A 
and  AN/ APQ--13,  form  B  ( PSa-B ). — This  check  is  designed  to  evalu¬ 
ate  aerial  proficiency  in  radar  navigation,  radar  bombing,  and  set 
operation.  This  check  is  administered  on  a  4  hour  mission  at  tho  con¬ 
clusion  of  a  student’s  aerial  training.  Throughout  this  chapter  it  will 
be  referred  to  as  the  final  aerial  check. 

Intermediate  aerial  performance  cheek  for  AN  /  APS-15,  AN/APS- 
lb  A ,  and  Aft/ 'APQ-13,  form  A  ( PSb-A ). — This  check  is  designed  to 
measure  a  student’s  aerial  proficiency  in  radar  navigation,  radar  bomb¬ 
ing,  and  set  operation  at  approximately  the  midpoint  of  his  aerial 
training.  The  navigation  and  bomb’  '  items  on  this  check  are  tho 
same  as  thoso  in  the  Final  Aerial  Check  except  that  coordinated 
bombing  procedure  is  not  checked  and  items  dealing  with  trouble- 
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shooting.  starting.  and  inning  arc  included.  Prior  to  1  June  1015 
[]iis  check  was  administered  to  only  one  student  on  a  single  flight 
, huinir  which  that  student  was  checked  on  three,  logs.  On  1  June 
]<»{.-,  and  thereafter  the  procedure  was  modified  so  that  two  students 
could  he  checked  on  a  single  flight.  Scores  from  the  more  recent  typo 
of  administration  have  hern  transformed  to  scores  comparable  to  those 
for  three  leys.  Throughout  the  chapter  this  check  is  referred  to  ns 
die  intermediate  aerial  check. 

1NTKRCORRKLATIONS  OF  TOTAL  SCORES  FOR  12  PROFI¬ 
CIENCY  MEASURES 

Table  S.l  presents  n  matrix  of  intercorrelations  for  12  proficiency 
measures.  Variables  1  through  f>  are  the  five  printed  tests  described 
above,  plus  a  score  obtained  by  totaling  Final  'Pests  I  and  II.  Vari¬ 
ables  7  through  12  arc  the  performance  checks.  All  of  the  cooflieients 
were  computed  by  the  Pearson  product-moment  method.2  The  table 
contains  essential  data  which  are  referred  to  and  interpreted,  for 
several  different  purposes,  throughout  the  remainder  of  the  chapter. 
It  is  inserted  here  to  give  a  general  background  for  the  specific  prob¬ 
lems  and  interrelations  which  will  bo  discussed. 

5 Tlu>  majority  of  t lie  Ktnllstlrnl  computations  were  made  l>y  T/St,’t.  Ilymnn  Rrlunlerer, 
S/S-t.  William  J.  Woywood,  C|d.  Owen  U.  Mimifer,  C|d.  Irving  Kmloninn,  and  Cpl.  James  R. 
Holt. 


Tahu;  SA.—fntrreorrcJaiions,  means,  and  standard  deviations  of  total  raw  scores 
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Of  (lie  Cl  coefficients,  10  are  significantly  dilTcmit  from  zero  at  the 
1-perccnl  confidence  level.  An  additional  11  are  signifi  ant  at  the 
5-percenl  level.  One  coefficient  is  "dive  but  is  not  significant. 

In  interpreting  table  8.1  two  factors  which  have  operated  in  un¬ 
known  strength  to  reduce  the  magnitude  of  the  obtained  coefficients 
should  he  kept  in  mind.  First,  the  11  proficiency  measures  were 
administered  at  various  times  in  the  course,  ranging  from  the  third 
week  to  the  tenth  week.  It  is  probable  that  Indents  progressed  at 
varying  rates  of  learning  during  the  periods  between  tests  or  cheeks. 
Members  of  the  Itadar  Project  enrolled  in  the  course  reported  that 
their  proficiency  improved  considerably  after  a  single  period  of  in¬ 
struction  from  a  good  instructor,  and  improved  slightly  or  not  at  nil 
over  long  periods  of  mediocre  instruction.  Second,  as  pointed  out  in 
clniplcin  5,  G,  and  7,  the  reliability  of  the  variables  correlated  may 
in  a  number  of  cases  be  cpiite  low.  It  was  not  always  possible  to  train 
lest  or  check  administrators  as  well  as  was  desired.  Examiners  ns 
well  as  the  students  were  often  not  well-motivated,  nor  were  adminis¬ 
trative  conditions  always  well  standardized.  However,  ns  already- 
noted,  it  was  impractical  to  compute  meaningful  reliability  coeffi¬ 
cients  and,  in  the  discussions  to  follow,  it  has  been  necessary,  conse¬ 
quently,  to  disregnrd  almost  entirely  attenuation  from  this  source. 

CO  It  It  ELATION  S  BETWEEN  PRINTED  TESTS  AND  PERFORM¬ 
ANCE  CHECKS  MEASURING  SIMILAR  SKILLS 

A  comparison  of  test  scores  with  scores  on  performance  checks 
which  nro  intended  to  measure  similar  skills  gives  an  indication  of 
the  extent  to  which  verbal  knowledge  of  basic  skills  is  related  to 
actual  performance.  If  the  two  are  closely  related,  a  printed  test 
which  can  be.  administered  to  a  large  group  of  students  at  one  time 
may  be  substituted  for  a  performance  check  wliuh  must  bo  adminis¬ 
tered  individually  mid  with  apparatus. 

Some  light  may  be  shed  upon  the  first  question  by  the  nnswer  to  a 
second.  IIow  do  the  correlations  tests  and  checks  compare  in  magni¬ 
tude  to  correlations  among  tests  and  to  those  among  checks?  If  the 
correlations  among  tests  and  those  among  checks  are  uniformly  high, 
n  relatively  unambiguous  answer  to  questions  involving  the  relations 
of  tests  to  checks  would  be  possible.  If,  on  the  other  hand,  the  cor¬ 
relations  of  either  test  with  test  or  check  with  check  nro  low,  less 
clear  interpretation  will  be  possiblo. 

A  third  question  is  concerned  with  tho  efTect  of  t ho  timo  of  ad¬ 
ministration  upon  the  correlations  of  tests  with  checks  having  similar 
content.  Arc  the  correlations  between  tests  and  checks  having  similar 
content  and  administered  at  the  same  time  in  the  course  higher  than 
the  correlations  between  equally  similar  tests  and  cheeks  administered 
ut  varying  times  in  the  course? 
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Answers  (o  (hose  questions  will  he  sought  in  (he  correlations  of  tho 
tulal  scores  of  tests  and  performance  checks,  in  the  correlations  of 
(t  t  scores  and  performance  check  part  scores,  and  in  the  correlations 
0f  more  inclusive  measures,  (he  proficiency  stanines. 

'fe-t  Scores  and  Performance  Check  Total  Scores 
Tho  correlations  between  scores  on  printed  tests  and  scores  on  per¬ 
formance  checks  arc  presented  in  table  8.2. 


T.uii.k  8.2.  — Relationships  of  printed  text  total  scores  to  performance  chcuk  total 
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Arc  tho  correlations  between  tests  and  checks  having  apparently 
similar  content  sufficiently  high  to  justify  dr'  arding  a  check  and  em- 
ploying  a  test  exclusively?  In  partial  answer  to  this  question  several 
of  the  relationships  in  table  8.2  may  be  noted. 

Final  Test  I,  a  printed  test  of  radar  navigation  skills,  is  related 
significantly,  at  tho  5-pcrccnt  confidence  level  or  better,  to  three  of  tho 
performance  checks.  Final  Test  I  correlates  significantly  at  tho  1- 
porcent  level  of  confidence  with  the  intermediate  aerial  check  and  at 
the  5-pcrccnt  level  of  confidence  with  the  two  intermediate  supersonic 
checks. 

Final  Test  II,  a  printed  test  of  knowledge  of  navigation,  bombing, 
end  especially  set  operation,  has  significant  relationships  with  two  of 
din  three  checks  which  are  not  significantly  related  to  Final  Test  I. 
Thoso  arc  the  final  supersonic  check  and  the  final  bench  set  check, 
which  is  essentially  a  performance  check  of  set  operation. 

Final  Test  I  and  Final  Test  II  in  combination  are  related  signifi¬ 
cantly  to  all  performance  checks  except  the  final  aerial  check.  In 
general,  the  combination  of  Final  Test  I  and  Final  Test  II  does  not 
ruisc  a  higher  relationship  with  any  particular  performance  check 
than  is  obtained  with  either  Final  Test  I  or  Final  Test  II  separately. 

The  set  operation  intermediate  test  correlates  significantly  only 
''‘fib  the  final  bench  set  check.  This  is  the  only  instanco  in  which 
si  mil  n  rity  of  content  appears  to  be  very  important  in  determining  tho 
relationship  of  a  printed  test  to  a  performance  check.  In  tad  !o  8..2 
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no  higher  relationship  is  found  between  two  bombing  measures  or 
between  two  navigation  measures  than  between  a  bombing  measure 
and  a  navigation  measure. 

Since  t lie  highest  relationship  between  a  printed  test  and  a  per¬ 
formance  check  measuring  similar  skills  is  represented  by  the  co¬ 
efficient  0.31,  between  the  set  operation  intermediate  te.-t  and  the 
final  bench  set  check,  it  is  clear  that  without  further  information  it 
would  not  be  feasible  to  discontinue  the  performance  checks  in  favor 
of  the  more  easily  administered  printed  tests.  As  already  indicated 
it  is  not  possible  to  estimate  the  effect  upon  the  correlations  discussed, 
of  the  low  reliability  which  may  characterize  the  performance  checks. 

IIow  do  the  correlat  ions  between  tests  and  checks  compare  in  magni- 
t  in  hi  to  the  correlations  among  tests  and  to  those  among  checks?  In 
table  S.l  significant  correlations  are  found  more  frequently  between 
pairs  of  printed  tests  than  between  printed  tests  and  performance 
checks.  Twelve,  or  02  percent  of  the  Id  iiuei correlat ious  among 
printed  tests  are  significant  at  the  5-percent  level  or  better,  whereas 
Id,  or  dG  percent,  of  the  dO  correlations  between  printed  tests  and 
checks  are  significant  at  the  5-percent  level.  Significant  correla¬ 
tions  are  found  with  equal  frequency  between  printed  tests  and  per¬ 
formance  cheeks  and  between  pairs  of  performance  checks.  In  Table 
S.l,  Id  or  d(5  percent,  of  the  3(>  correlations  between  printed  tests  and 
checks  are  significant,  ns  compared  to  5  significant  coefficients,  or  33 
percent,  among  the  15  correlations  between  checks. 

Arc  the  correlations  between  tests  and  checks  having  similar  con¬ 
tent  and  administered  at  approximately  the  same  period  in  the  course 
higher  than  the  correlations  between  tests  and  checks  administered  at 
varying  times?  There  is  no  indication  in  table  8.2  that  this  is  the 
case.  The  combined  score  for  Final  Tost  I  and  Final  Test  II,  given 
in  the  tenth  week,  correlates  more  highly  with  the  intermediate  aerial 
check  given  in  the  sixth  week  than  with  the  final  aerial  check  given  the 
ninth  or  tenth  wcclc,  and  just  about  as  highly  with  bombing  super¬ 
sonic.  intermediate  check  (fifth  week)  or  the  navigation  supersonic 
intermediate  check  (third  week)  ns  with  the  final  supersonic  check 
(seventh  week). 

Test  Scores  and  Performance  Check  Part  Scores 

The  final  aerial  check  and  final  supersonic  check  both  contain  items 
on  radar  navigation  and  radar  bombing.  The  inclusion  of  two  skills 
in  n  single  performance  check  might  attenuate  any  correlation  of  that 
check  with  printed  tests  designed  to  measure  only  one  of  the  two  skills. 
Therefore,  the  navigation  items  and  the  bombing  items  of  tho  two 
clucks  were  separately  scored  and  the  part  scores  wore  correlated  with 
printed  tests  considered  to  he  homogeneous  tests  of  navigation  or  of 
bombing.  In  the  case  of  the  final  supersonic  checks,  the  navigation 
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items  find  the  bombing  items  of  each  of  the  two  forms  were  correlated 
'with  each  of  the  printed  tests,  and,  using  the  Fisher  3-function  trans¬ 
formation,  the  coefficient  between  form  A  of  the  check  and  a  given 
printed  test  was  averaged  with  that  between  form  B  and  the  test. 
The  correlation  coefficients  and  other  relevant  data  are  found  in  table 
8.3. 


Tabi.b  8.3. — Relationships  of  printed  test  total  rato  scores  to  performance  check 

part  scores' 
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i  Oocfliclents  In  these  columns  arc  averages  of  coefficients  for  form  A  and  form  D  of  the  finrl  supcrsonlo 
check-.  Tlio  Fisher  7.  correlation  function  was  employed  In  computing  theso  averages. 


None  of  tho  coefficients  in  table  8.3  is  significant  at  the  5-perccnt 
level  of  confidence.  When  bombing  or  navigation  test  scores  are 
correlated  with  similar  bombing  or  navigation  part  scores  for  the  final 
aerial  check  and  tho  final  supersonic  check,  tho  coefficients  obtained 
are  somewhat  loAvcr  than  thoso  obtained  with  the  total  scores  for 
theso  checks.  The  latter  coefficients  appear  in  table  8.2.  This  decre¬ 
ment  in  relationship  may  bo  duo  to  the  lower  reliability  associated 
with  tho  smaller  number  of  items  in  the  part  scores  as  compared  with 
tho  total  scores. 

Proficiency  Slnnincs 

Three  composite  proficiency  measures  were  computed,  one  based 
upon  printed  classroom  tests,  ono  upon  trainer  performance  checks, 
and  the  third  upon  aerial  performance  checks.  Each  has  been  con¬ 
verted  to  stanine  form.  Tlw,  classroom  stanine  is  based  on  Final  Test 
I  and  Final  Test  II,  which  together  are  weighted  three,  and  Radar 
Navigation  Intermediate  Test,  which  is  weighted  one,  the  Radar 
Bond »ing  Intermediate  Test,  weighted  one,  and  the  Set  Operation 
Intermediate  Test,  weighted  one.  Tho  trainer  stanino  is  based  on 
t ho  Final  Supersonic  Check,  weighted  three,  the  Navigation  Super¬ 
sonic  Intermediate  Check,  weighted  one,  tho  Bombing  Supersonic 
Intermediate  Check,  weighted  one,  and  the  Final  Bench  Set  Check, 
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weigh  toil  one. 


The  aerial  staniiie  is  based  on  the  Filial  Aerial  Check 
weighted  two,  and  the  Intermediate  Aerial  Cheek  weighted  one.  In 
computing  each  of  the  three  stanines,  the  individual  scores  were  fn^t 
converted  to  standard  scores,  then  weighted  and  averaged. 

The  intereorrelations  of  these  three  proficiency  stanines  aro  pre¬ 
sented  in  table  8.4.  The  correlations  of  the.  classroom  stanine  with  the 


Tahijc  8.1. — Intereorrelations  of  proficiency  stanines 
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trainer  and.  aerial  stanines  are  significant  at  the  1  percent  level.  The 
relationship  of  the  classroom  stanine  to  the  trainer  stanine  may  bo 
somewhat  higher  than  it  is  to  the  aerial  stanine  although  the  difference 
is  not  statistically  significant.  The  correlation  of  the  trainer  stanine 
with  the  aerial  stanine  is  not  significant.  In  interpreting  these  ob¬ 
served  relationships  the  possibility  of  low  reliability  for  the  trainer 
and  aerial  stanines  must  be  kept  in  mind. 

CORRELATIONS  BETWEEN  PAIRS  OF  TESTS  AND  BETWEEN 

PAIRS  OF  PERFORMANCE  CHECKS  MEASURING  SIMILAR 
SKILLS 

This  section,  like  the  one  preceding  it,  is  concerned  with  tho  inter¬ 
relations  of  proficiency  scores  measuring  similar  skills.  The  analysis 
presented  is  based  upon  correlations  between  pairs  of  printed  tests 
or  pairs  of  performance  checks.  All  total  scores  or  part  scores  have 
been  used  which  clearly  measure  one  of  three  basic  skills  of  the  radar 
observer:  radar  navigation,  radar  bombing,  or  set  operation.  Corre¬ 
lations  between  measures  of  different  skills  have  been  included  in  the 
tables  to  allow  comparison  with  correlations  between  measures  of  the 
same  skill. 

Comparisons  will  be  made  with  the  objective  of  answering  specific 
questions  such  as  the  following.  Do  measures  of  similar  skills  corre¬ 
late  higher  than  measures  of  dissimilar  functions?  Do  printed  tests, 
trainer  checks  or  aerial  checks  yield  the  highest  interrelations?  Of 
the  three  basic  skills  studied,  which  yields  the  highest  intercorrela- 
(ious?  Does  the  time  in  the  course  at  which  a  test  or  check  was  admin¬ 
istered  affect  its  correlation  with  other  scores? 


CORRELATIONS  BETWEEN  PERFORMANCE  CHECKS  MEAS¬ 
URING  SIMILAR  SKILLS 

Tho  correlations  in  tablo  8.5  suggest  that  so  far  as  the  supersonic 
trainer  is  concerned  measures  of  similar  functions  do  not  correlate  ’ 
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more  highly  than  measures  of  dissimilar  functions.  The  Navigation 
Supersonic  Intermediate  Check  correlates  significantly  with  the  bomb¬ 
ing  items  and  with  the  total  score  of  the  Final  Supersonic  Check;  it 
does  not  correlate  significantly  with  the  navigation  items.  On  the- 
other  hand,  the  Bombing  Supersonic  Intermediate  Check  irf  almost- 
equally  related  to  the  navigation  part  score,  the  bombing  part  score*, 
and  the  total  score  of  the  Final  Supersonic  Check. 

Correlations  between  part  and  total  scores  of  supersonic  trainer 
performance  checks  and  part  scores  of  the  Final  Aerial  Check  are  pre¬ 
sented  in  table  8.G.  Since  only  one  coellicient  in  tho  table  is  signili- 

TAm.B  8.0. — Correlations  of  supersonic  check  raio  scores  tcllh  navigation  ami 
bombing  Hems  of  the  final  aerial  check 
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cant  at  the  5-pereeiit  level  no  significant  comparisons  are  possible.  The 
correlation  of  0.23  between  the  Bombing  Supersonic  Intermediate 
Cl  ieek  and  bombing  items  in  the  Final  Aerial  Check  may  be  taken  as 
slight  evidence  that  bombing  performance  on  the  supersonic  trainer 
is  more  closely  related  to  aerial  bombing  performance  than  naviga¬ 
tion  performance  on  the  trainer  is  related  to  aerial  navigation 
performance. 

Table  8.7  summarizes  the  correlations  of  total  scores  on  the  super¬ 
sonic  trainer  performance  checks  with  total  scores  on  the  aerial  per- 


T.uiUi  8.7. — Intcrcorrclations  of  supersonic  cheek  raw  scores  t cith  total  raw 

scores  on  aerial  checks 
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formance  checks.  Tho  correlation  of  0.18  between  tho  Final  Super¬ 
sonic  Check  and  the  Intermediate  Aerial  Check  is  the  only  correlation 
significant  at  the  5-perccnt  level.  These  two  performance  checks  were 
administered  within  a  week  of  each  other  and  are  both  combined 
measures  of  navigation  and  bombing. 

Correlations  Between  Tests  Measuring  Similar  Skills 
Tablo  8.8  presents  the  intercorrelations  of  total  scores  of  all  printed 
tests.  Of  tho  13  correlations  in  this  tabic,  11  are  significant  at  tho  1- 


Taiiij-:  8.8. — Intcrcorrclations,  means,  and  standard  deviations  of  total  raw  scores 
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percent  level  ami  another  at  the  5-percent  level  of  confidence.  Final 
Test  I  correlates  more  highly  with  the  other  navigation  measure,  (ho 
Radar  Navigation  Intermediate  Test,  than  with  any  other  test.  Tho 
lattei  test,  however,  correlates  about  as  well  with  the  dissimilar  Radar 
Bombing  Intermediate  Test  as  it  does  with  the  similar  Final  Test  I. 
The  Set  Operation  Intermediate  Test  correlates  more  highly  with 
Final  Test  II,  which  is  heavily  weighted  with  set  operation  items,  than 
with  any  dissimilar  test.  On  tho  whole,  table  8.8  gives  only  slight 
evidenco  for  the  expected  result,  that  printed  tests  measuring  tho  sarno 
skills  are  more  closely  interrelated  than  printed  tests  measuring  dif¬ 
ferent  skills. 

Since  Final  Test  II  measures  all  three  basic  skills,  part  scores  for 
this  test  have  been  correlated  with  total  scores  of  tests  considered  to 
bo  relatively  homogeneous  measures.  The  correlations  are  presented 
in  table  8.9.  Differences  in  tho  obtained  coefficients,  while  in  tho 
expected  direction,  are  not  of  great  magnitude. 

Table  8.0. — Correlations  between  Intermediate  testf  and  part  scores  of  Final 

Test  II 
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Certain  additional  data  regarding  the  relationships  among  measures 
of  similar  skills  arc  available  which  are  not  presented  in  tabular  form. 
Correlations  were  computed  between  parts  of  the  Radar  Navigation 
Intermediate  Tests  and  corresponding  parts  of  Final  Tests  I  and  IT. 
Scores  of  177  Langley  students  on  the  airplot  and  E-OB  sections  of 
tho  Radar  Navigation  Intermediate  Test  correlated  0.30  with  scores  on 
tho  airplot  and  E-GB  computer  sections  of  Final  Test  I.  Scores  of 
1S3  Langley  students  on  Final  Test  II  navigation  items  correlated  0.12 
with  their  scores  on  similar  items  in  the  first  part  of  the  Radar  Navi¬ 
gation  Intermediate  Test.  While  these  results  suggest  that  printed 
measures  of  navigational  knowledge  arc  more  closely  related  than 
measures  of  navigational  performance,  it  is  equally  likely  that  they 
only  reflect  differences  in  reliability. 

In  Minimary,  measures  of  the  same  skill,  as  skill  is  here  defined, 
seem  very  slightly  if  at  all  more  closely  related  than  measures  of 
different  skills.  Printed  tests  furnish  the  highest  intercorrelations 
whethor  tho  skills  correlated  nro  similar  or  dissimilar.  Of  the  three 
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basic  skills,  then',  i.s  some  evidence  in  t ! ic  correlations  between  per¬ 
formance  cheeks  that  the  bombing  measures  are  more  closely  related 
than  either  navigation  or  set  operation  measures,  although  there  are 
few  data  involving  the  set  operation  measures.  The  Bombing  Su¬ 
personic  Intermediate  Cheek  is  more  closely  related  to  the  bombing 
items  in  the  Final  Supersonic  Check  than  the  Navigation  Supersonic 
Intermediate  Check  is  related  to  the  navigation  items  in  the  Final 
Supersonic  Cheek.  Also,  the  Bombing  Supersonic  Intermediate  Cheek 
is  more  closely  related  to  the  bombing  items  in  the  Final  Aerial 
Cheek  than  the  Navigation  Supersonic  Intermediate  Check  is  related 
to  navigation  items  in  the  Final  Aerial  Check.  With  regard  to 
the  inti uencc  of  time  of  administration  the  evidence  from  tables  8.5 
through  8.9  indicates  that  the  period  of  the  course  during  which  tho 
proficiency  measures  were  administered  has  no  consistent  c fleet  on 
the  magnitude  of  their  relationship. 


COIUt ELATIONS  BETWEEN  RATINGS,  PERFORMANCE 
CHECKS,  AND  FINAL  TESTS 


It  was  noted  in  the  preceding  section  that  the  correlations  between 
scores  on  tho  various  performance  cheeks  were  uniformly  low.  This 
finding  suggested  tho  desirability  of  correlating  instructors’  ratings  of 
performance  on  these  same  checks  on  the  hypothesis  that  such  corre¬ 
lations  might  be  higher.  In  formulating  this  hypothesis  it  was  rea¬ 
soned  that  ratings  might  not  only  take  account  of  proficiency 
demonstrated  during  tho  check  ride,  but  also  allow  for  tho  variable 
conditions  under  which  tho  student  had  to  work.  If  ratings  were 
independent  of  such  variations  while  performance  cheek  scores  were 
influenced  by  them,  correlations  of  the  former  would  be  expected  to 
reveal  more  faithfully  the  magnitude  of  any  true  relationship. 

The  ratings  correlated  were  made  by  examiners  at  the  completion 
of  their  administration  of  performance  cheeks.  Becauso  the  exam¬ 
iners  were  accustomed  to  thinking  in  terms  of  centilc  grades,  the  scale 
used  was  divided  into  10  equal  divisions,  each  supposedly  containing 
10  percent  of  the  students.  Instructions  called  for  making  tho  rating 
in  relation  to  all  students  to  whom  the  examiner  had  given  perform¬ 
ance  checks  and  on  the  basis  of  how  well  he  thought  the  student  would 


have  done  under  normal  conditions. 

Product-moment  correlations  wero  computed  between  ratings  on 
performance  cheeks  and  between  such  ratings  and  the  final  proficiency 
tests.  Of  tho  correlations  computed,  only  two  aro  significant  at  tho 
5-percent  level.  Killings  on  the  Final  Aerial  Check  for  71  Langley 
students  correlate  0.22  with  ratings  on  the  Final  Supersonic  Check. 
The  correlation  between  scores  on  the  corresponding  performance 
checks  for  190  Langley  students  is  presented  in  table  8.1 ;  the  coefficient, 
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0.1!,  is  not  significant.  The  correlation  between  ratings  on  the  Final 
Aerial  Check  and  the  combined  scores  on  Final  Te.-ts  l  and  II  for  71 
Langley  students  is  0.23.  As  indicated  in  table  o.l,  the  Final  Aerial 
Check  correlated  0.11  with  the  scores  of  100  Langley  indents  on  the 
combined  final  proficiency  tests.  The  latter  coefficient  is  not  signif¬ 
icant.  Additional  correlations  we  e  computed  between  ratings  for  75 
Victorville  students  on  the  Intermediate  Aerial  Check  and  the  Final 
Aerial  Check  (-0.17),  between  ratings  for  GO. Langley  students  on  the 
Intermediate  Aerial  Check  and  the  Final  Supersonic  Check  (0.12); 
and  between  ratings  on  Final  Aerial  Check  and  scores  on  Final  Tests 
I  and  II  for  320  Victorville  students  (0.0G).  None  of  these  reached 
the  5  percent  level  of  significance. 

CORRELATIONS  BETWEEN  NAVIGATION,  SET  OPERATION, 

AND  BOMBING  SKILLS 

Tho  curriculum  assigned  approximately  equal  weight  to  radar  navi¬ 
gation,  radar  bombing,  and  radar  set  operation.  In  the  classroom, 
10  hours  of  lccturo  time  are  devoted  to  radar  navigation,  40  hours  to 
radar  bombing,  and  20  hours  to  set  operation,  although  instruction  in 
the  latter  is  also  included  in  both  radar  navigation  and  bombing.  Set 
operation  instruction  on  the  trainer  consists  of  10  hours  on  the  bench 
sets.  Combined  navigation  and  bombing  instruction  is  given  during 
28  hours  of  supersonic  training  and  80  hours  of  aerial  training.  In 
almost  all  instances,  however,  the  three  categories  overlap.  Naviga¬ 
tion  and  bombing  skills  may  hardly  be  taught  or  applied  apart  from 
set  operation  skill.  The  set  must  be  readjusted  frequently  to  satisfy 
varying  navigational  and  bombing  requirements.  Consequently, 
measures  of  bombing  and  navigation  skill  tend  to  be  influenced  by 
skill  at  set  operation.  Also,  navigation  and  bombing  overlap.  Mak¬ 
ing  heading  corrections  on  tho  bombing  run  is,  in  a  sense,  merely 
precision  navigation. 

From  such  considerations  ns  these,  it  is  expected  that  test  scores 
measuring  these  three  different  skills  will  be  interrelated  in  some 
degree.  The  comparisons  to  be  made  will  fall  in  five  different  cute- 
gm-if's:  Part  of  a  test  against  another  part  of  the  same  test,  part  of 
a  performance  chock  against  another  part  of  the  same  cheek,  test 
against  test,  performance  check  against  performance  check,  ar.d  test 
against  performance  chock.  Within  these  categories  it  should  be  pos¬ 
sible  to  make  comparisons  between  correlations  based  upon  pairs  of 
measures  with  roughly  equivalent  reliabilities. 

Part  Against  Part  of  the  Same  Test 

<> 

Section  A  of  Final  Test  If  consists  of  83  bombing  and  22  naviga¬ 
tion  items.  For  the  Langley  sample  of  120  students,  the  correlation 
is  0.50  between  scores  from  Iho-o  two  groups  of  items.  When  odd 
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Part  Against  Part  of  the  Same  Performance  Cheek 

For  the  same  sample  of  100  Langley  students.  Final  Aerial  Check 
r  avigation  items  correlate  0.50  with  bombing  items.  In  the  Final 
Supersonic  Check,  navigation  items  correlate  0.35  with  the  bombing 
items.  As  was  done  above,  these  correlations  may  be  compared  to 
others  in  which  one  mixed  bombing-navigation  score  is  correlated 
with  another.  For  the  Final  Aerial  Check,  for  example,  the  correla¬ 
tion  is  0.17  between  legs  1,  3,  and  5,  and  legs  2  and  4.  No  parallel 
con  elation  is  available  for  the  Final  Supersonic  Check. 

The  first  and  second  legs  of  the  Navigation  Supersonic  Intermedi¬ 
ate  Check  correlate  0.30,  and  the  first  and  second  legs  of  the  Bombing 
Supersonic  Intermediate  Check  correlate  0.47. 

It  is  suspected  that  halo  effect  operated  to  increase  the  magnitude 
of  the  correlations  reported  in  the  above  paragraphs,  since  the  scores 
correlated  are  assigned  by  the  same  instructor  in  a  single  testing 
p m  iod.  The  greater  halo  effect  for  the  aerial  check  than  for  the  more 
s  tandardized  supersonic  check  is  in  the  expected  direction. 

Test  Against  Test 

Correlations  between  radar  navigation  and  radar  bombing  tests  are 
presented  in  table  8.10.  It  is  apparent  that  skill  in  navigation  is 
positively  correlated  with  skill  in  bombing,  at  least  as  measured  by 
printed  tests. 


Tams.  8.10. — Correlations  bctwr.cn  tests  measuring  radar  navigation  and  those 

measuring  radar  bombing 
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Three  correlations  arc  available  relating  set  operation  skill  to  radar 
navigation  and  bombing  skills.  The  Set  Operation  Intermediate  Test 
correlates  0.10  with  Final  Test  I,  a  navigation  measure,  and  0.14  with 
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(ho  Navigation  Intermediate  Tost,  'l'ho  lattor  figure  is  significant. 
a(  the.  f»-pereent  lovol.  The  Sot  Operation  Intermediate  Tost  and 
Kailar  Bombing  Intermediate  'I'ost  correlate  0.80,  significant  at  t ho 
1-pereent  level. 

In  summary,  the  results  for  printed  tests  indicate  that,  bombing 
skill  is  significantly  related  to  both  navigation  and  sot  operation 
skills.  Correlations  between  tests  of  those  skills  range  from  0.28  to 
O.r.O  or  higher.  The  relationship  of  navigation  to  set  operation  is  low 
and  of  questionable  statistical  significance. 


Performance  Check  Against  Performance  Chock 
Table  8.11  presents  the  correlations  between  performance  chocks 
measuring  radar  navigation  and  those  measuring  radar  bombing. 
Three  of  tho  seven  correlations  are  significant  nt  the  5-poreont  level. 
All  arc  between  navigation  and  bombing  items  from  supersonic  chocks. 


Taiji.k  8.11. — Correlations  5o/iror»  performance  checks  ineasn rh  g  radar  tiaviga- 
tion  and  those  measuring  radar  bombing 
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From  other  tables  two  correlations  are  available  between  f. t  oper¬ 
ation  skill  as  measured  by  performance  cheeks  and  radar  navigation 
and  bombing.  The  Final  Bench  Bet  Check  correlate;;  0.11  with  tho 
Navigation  Supersonic  Intermediate  Check  and  O.l.'J  with  the  Bomb¬ 
ing  Intermediate  Check,  as  indicated  in  table  8.1.  Neither  of  the"© 
cociiieients  is  significant  at  the  5 -percent  level. 

Tcit  Against  Performance  Check 

In  table  8.12,  none  of  the  correlations  between  navigation  and  set 
operation  skills  or  between  bombing  and  set  operation  are  Tgnificant 
at  the  5 -percent  level.  Of  the  nine  correlations  between  navigation 


mid  bombing  measures  one  is  significant  at  the  1-percent  level,  and  an¬ 
other  at  the  5-pcrccnt  level.  The  Eadar  Bombing  Intermediate  Test 
correlates  slightly  higher  with  the  Navigation  Supersonic  Inter¬ 
mediate  Cheek  (r  —  O.iM)  than  with  the  Bombing  Supersonic  Inter¬ 
mediate  Check  (r.  =  .l7,  table  8.1).  The  Radar  Navigation  Interme¬ 
diate  Test  correlates  slightly  higher  with  the  Bombing  Supersonic 
Intermediate  Check  (?■-- 0.18)  than  with  the  Navigation  Supersonic 
Intermediate  Check  (;•=—  0.01,  table  8.1).  This  comparison  seems 
again  to  confirm  the  view  that  bombing  skill  overlaps  considerably 
with  navigation  skill. 

'J' a  lira:  8.12.-  -Correlations  between  printed  tests  and  performance  cheeks  meas¬ 
uring  radar  bombing,  radar  navigation,  and  set  operation 
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Insofar  as  (lie  printed  tests  and  performance  cheeks  are  valid  meas¬ 
ures  of  the  three  basic  skills  examined  in  this  section  of  the  chapter, 
it  appears  that  navigation  and  bombing  skills  are  the  most  closely 
related,  bombing  and  set  operation  skills  arc  somewhat  less  closely 
related,  and  navigation  and  set  operation  skills  are  least  closely  related. 


CORRELATIONS  BETWEEN  PART  SCORES  OK  FINAL  TEST  I 


Final  Test  I,  a  measure  of  radar  navigation  skills,  consists  of  four 
sections.  Section  A  is  a  simulated  navigation  and  bombing  mission 
involving  the  use  of  simulated  scope  photos,  a  radar  Mercator  chart 
of  Germany,  an  IC-GB  computer,  dividers,  and  Weems  plotter.  Sec¬ 
tions  B,  C,  and  D  arc  measures  of  the  three  primary  skills  involved  in 
section  A.  Section  B  requires  the  student  to  plot  aircraft  position 
from  20  simulated  scope  photos.  Section  C  consists  of  20  IC-GB  com¬ 
puter  problems.  Section  D  consists  of  20  airplot  problems.  It  was 
expected  that  inter 'correlations  between  sections  B,  C,  and  D,  which 
were  included  in  the  test  in  order  to  provide  added  length  and  reliabil¬ 
ity,  would  be  fairly  high.  It  was  expected  also  that  correlations 
between  section  A  and  each  of  the  other  sections  would  be  high,  with 
the  correlation  between  section  A  and  the  sum  of  sections  B,  C,  and 
D  being  the  highest  of  all.  These  correlations  arc  presented  in  table 
8.13.  All  arc  significant  at  the  1  percent  level  of  confidence.  The 
relationship  of  the  mission  to  its  component  parts  is  somewhat  lower 
than  the  relationship  of  the  component  parts  to  each  other.  The  re¬ 
lationship  of  the  mission , to  the  sum  of  its  component  parts  :s,  sur¬ 
prisingly,  lower  than  the  relationship  of  the  component  parts  to  each 
other. 


T.v.»r.K  S.13. — Intcrcorrclntions  of  four  sections  of  Final  Test  /  tria-ll)  based 
on  Langley  Field  classes  -J 5-19  through  }J-33 
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There  is  some  reason  to  believe,  however,  that  those  results  are 
peculiar  to  the  sample  upon  which  intwcorrelutions  in  this  chapter 
urc  based.  Two  similar  studies  were  made  previously,  one  of  2GG 
students  in  classes  11-12  through  -l  1-1G  at  Langley  Field,  and  another 
of  309  students  in  classes  -ia-S  through  !  “>—  1  -i  at  Langley  Field.  Those 
results  lire  presented  in  tables  8.M  and  8.15. 
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Comparison  of  tables  8.13,  8.14,  and  8.15  shows  that  tho  interrela¬ 
tionships  between  sections  B,  0,  and  D  remain  approximately  the 
same  in  each  of  the  three  studies,  fn  Tables  8.1 1  and  8.15,  however, 
the  simulated  mission,  section  A,  correlates  as  high  with  each  of  its 
component  parts,  B,  C,  and  D,  as  do  these  component  parts  with  each 
other.  Moreover,  in  both  these  tables  the  highest  correlation  is  that 
of  section  A  with  the  sum  of  section  B,  C,  and  I).  The  results  are  very 
similar  to  wlmt  was  originally  anticipated. 


T.wu.r.  8.11. — Inlcrcorrelallona  of  tho  four  sections  of  Final  Test  I  ( Pla-Il ) 
bused  on  Lunglcy  Field  classes  t-i-12  through  Jrf-IG 
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Taumj  8.15. — Intcrcorrclallons  of  the  four  sections  of  Final  Test  I  ( Pla-B ) 
based  on  Langley  Field  classes  45-8  through  45-1 J 
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A  possiblo  explanation  for  tho  differences  in  results  in  tho  three 
tables  lies  in  differences  in  tho  samples.  Tho  snmplo  in  table  8.18 
consists  almost  entirely  of  bombardiers,  whereas  the  samples  in  tablo 
8.M  and  table  8.15  consists  of  approximately  two-thirds  navigators  and 
one-third  bombardiers.  Navigators,  by  reason  of  their  greater  experi¬ 
ence  with  navigation  on  aerial  and  trainer  missions,  will  have  had 
considerably  nioro  practico  than  bombardiers  in  tho  planning  and 
organizational  requirements  of  the  simulated  mission  type  of  problem. 
For  them  it.  is  possible  that  section  A  constitutes  a  different  test  than 
for  bombardiers.  If,  for  tho  navigators,  the  organizational  features 
of  navigation  havo  become  routine,  it  would  bo  likely  that  their  per¬ 
formance  on  tho  simulated  mission  would  be  determined  in  good  part 
by  their  skill  nt  operations  tested  in  sections  B,  C,  and  D.  The  corre¬ 
lations  of  these  sections  with  section  A  would  be  correspondingly  large. 
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If,  for  the  bombardiers  of  table  8. Id,  on  tbo  other  hand,  the  simulated 
mi:  ion  presented  new  and  unfamiliar  problems,  (he same  relationships 
might,  he  lower. 

RELATED  REPORTS  FROM  TIIE  AVIATION  PSYCHOLOGY 

PROGRAM 

Since  the  training  of  the  radar  observer  is  similar  to  that  of  navi¬ 
gators  mid  bombardiers,  it  is  of  interest  to  compare  intercorrelations  of 
proficiency  measures  reported  here  with  those  reported  elsewhere  by 
Psychological  Research  Project  (Bombardier)  and  Psychological 
Research  Project  (Navigator).  Correlations  from  other  projects  are 
available  for  printed  tests  with  aerial  performance  checks,  and  printed 
tests  with  other  printed  tests. 

Printed  Tests  Against  Aerial  Performance  Checks 

l'wo  types  of  printed  tests  have  been  correlated  with  aerial  per¬ 
formance  checks.  In  one  type  each  item  is  a  separate  and  independent 
measure  of  some  knowledge  or  skill;  the  other  type  attempts  to  simu¬ 
late  an  aerial  mission  and  to  measure  a  series  of  related  tasks.  In  the 
latter  type,  the  items  of  the  test  are  interdependent.  The  Bombardier 
Project  has  correlated  the  first  type  of  printed  test  with  an  aerial 
•heck.  The  Navigator  and  Radar  Projects  report  correlations  for 
both  types  of  tests. 

The  Bombardier  Proficiency  Test  (form  B)  is  a  3-hour  examination 
providing  a  comprehensive  coverage  of  basic  bombing  subjects.  Tho 
Bombardier  Project  has  developed  standardized  phase  checks,  which 
are  aerial  performance  checks.  For  107  cadets,  the  Bombardier  Pro¬ 
ficiency  Test  correlates  0.19  with  Phase  Check — Form  1.  For  177 
instructors  from  all  bombing  schools  the  tost  correlates  0.33  with  Fhaso 
Cheek — Form  3. 


T a n r.K  8.10. — Correlations  of  printed  tests  measuring  navigation  proficiency  with 

the  navigation  flight  mission 1 
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.  .1.11,’.  l.il’i'l  o.'t.  p.  49. 

i  ’|  i  ’  it  ■  i ; ■  ■ . >  n,l-. n  I-,  n  prlntm  t(  4  which  simulates  nn  aerial  mission. 

• :  mu  .nt  ia  ihu  1  ivin  lit  level  of  0>ufli|rnvo. 

The  Navigator  Proficiency  Test  (form  A)  consists  of  eight  parts 
dealing  with  various  aspects  of  navigation,  including  technical  vocab- 
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ulary,  theory,  and  procedure.  The  flight  mission  developed  by  the 
Navigator  Project  is  a  standardized  cheek  of  aerial  performance;  the 
ground  mission  is  a  printed  Lest  simulating  an  aerial  mission.  Corre¬ 
lations  of  the  flight  mission  with  the  proficiency  lest  and  the  gronnd 
mission  are  presented  in  table  8.10.  The  "round  mission  is  correlated 
Significantly  with  the  flight  mission  while  the  proficiency  test  is  not. 
Comparable  *  orrelations  between  radar  printed  tests  and  performance 
checks  have  been  dismissed  earlier  in  this  chapter.  They  are  sum¬ 
marized  for  comparison  in  table  8.17. 


Tahi.1-:  S.‘.7. — Correlation*  of  printed  text*  measuring  radar  navigation  and  radar 
bombing  with  aerial  performance  checks 
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Correlations  Iletwc.cn  Printed  Tests 

The  Navigator  Project  reports  intcreorrelations  between  the  Navi¬ 
gator  Proficiency  Test  (form  A),  the  ground  mission,  and  a  written 
examination  containing  items  of  information  about  basic  navigation. 
The  saiuplo  consisted  of  201  navigation  students.  The  Navigator 
Proficiency  Test  (form  A)  and  the  written  examination  correlate 
0.08.  The  Navigator  Proficiency  Test  (form  A)  and  tho  ground 
mission  correlate  0.29.  The  ground  mission  and  the  written  exami¬ 
nation  correlate  0.4 1.3 

The  Bombardier  Project  reports  intcreorrelations  of  the  Bombardier 
Proficiency  Test  (form  B)  described  in  the  preceding  section,  the  Ra¬ 
dar  Navigation  Test  P-5B,  tho  Bombardier  Proficiency  Test  (form 
C),  First  Booklet,  and  tho  Bombardier  Proficiency  Test  (form 
CN).  The  Radar  Navigation  Test  P-5B  is  n  printed  test  of  basic 
navigation  designed  to  select  students  for  radar  observer  training. 
Bombardier  Proficiency  Test  (form  C),  First  Booklet,  is  a  test  dealing 
with  basic  bombing  subjects  and  weather.  Bombardier  Proficiency 
Test  (form  CN)  is  a  basic  navigation  test. 

For  a  sample  of  182  student  bombardiers,  the  Bombardier  Proficiency 
Test  (form  B)  and  the  Radar  Navigation  Test  P-f)B  correlate  0.51. 
The  Bombardier  Proficiency  Test  (form  B)  and  Bombardier  Pro- 

>  Si'i>  Curler,  I„  K.,  at.  P/tgrbologlcal  rcHcarch  on  navigator  training.  AAF  nvlfttlon 
jikvi-IioIii,-;)'  program  research  reports,  uo,  10.  Washington  :  Government  lYIntlng  Office, 
1047. 


164 


liciency  Test  (form  CN)  also  correlate  0.51.  The  Bombardier  Pro- 
[Fmaey  Test  (form  B)  and  Bombardier  Proficiency  Tc.-.t  (form  C), 
l-'i r:  t  Booklet,  correlate  0.57.  The  sample  for  this  coefficient  was  iil'J 
v!  util'll ts. 

Table  8.8  of  (his  report  gives  intercorrelations  of  printed  te-ts  u  ed 
by  the  Radar  Project.  For  all  three  projects,  the  correlations  between 
printed  test  scores  are  generally  higher  than  the  correlations  between 
printed  test  scores  and  aerial  performance  scores.  Both  the  Bom¬ 
bardier  Project  and  t ho  Radar  Project  obtained  correlations  between 
bombing  tests  and  navigation  tests  which  are  only  slightly  lower  than 
(he  correlations  between  two  bombing  tests  or  two  navigation  test.s. 
Tone  of  the  projects  found  the  correlation  between  printed  tests  and 
aerial  performance  checks  to  be  high. 

SUMMARY 

The  chapter  presents  and  interprets  correlations  between  scores 
on  printed  tests  and  performance  checks.  Since  the  reliabilities  of 
the.  measure  reported  are  unknown,  the  analyses  made  are  subject 
to  qualification  and  are  regarded  as  tentative  only. 

For  purposes  of  analysis,  the  radar  observer’s  task  is  broken  down 
into  three  basic  skills:  radar  navigation,  bombing,  and  set  operation. 
A  fourth  skill,  oscilloscope  interpretation,  is  not  considered  in  this 
chapter  because  no  proficiency  tests  or  checks  were  developed  for  its 
measurement.  One  of  the  questions  raised  concerned  the  practicability 
of  substituting  tests  for  performance  checks  which  arc  less  easily  ad¬ 
ministered  and  which  require  apparatus.  Xo  support  of  this  possi¬ 
bility  may  he  gained  from  the  obtained  correlations.  Most  coothcients 
are  not  significantly  different  from  zero.  The  highest  correlation  ob¬ 
tained  between  a  test  and  check  measuring  similar  skills  is  0.81  be¬ 
tween  the  Set  Operation  Intermediate  Test  and  the  Final  Bench  Set 
(  heck.  It  is  recognized  that  this  lack  of  relationship  might  be  a  con¬ 
sequence  of  the  unreliability  of  the  measures  involved. 

Printed  tests  are  closely  related  to  performance  checks  as  perform¬ 
ance  checks  are  to  each  other.  Higher  than  either  are  the  interoorrel.v 
tions  among  printed  tests.  This  dilfercmv  holds  regardless  of  the 
:  imilarity  or  dissimilarity  of  the  skills  correlated.  Printed  tests  which 
simulate  radar  observer  skills  a  re  somewhat  more  closely  related  to 
performance  chocks  than  arc  printed  tests  of  technical  information. 
Thm-e  is  no  indication  that  printed  tests  correlate  more  highly  with 
Bweks  given  at  the  same  period  in  the  course  than  with  those  given 
at  a  diiferent  period. 

Tests  and  performance  checks  measuring  similar  skills  are  found  to 
l  e  related  about  as  strongly  as  are  measures  of  unlike  skills.  Bombing 
:  cores  are  more,  closely  interrelated  than  are  navigation  measures. 
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The  relation  of  lx>:: iV>:. r-*. lt  ar.d  mvig  ition  .-cores  to  set  <  n.'iv.t  i<m  '-■’cres 
i ;  not  i  1 1 v e.- * i  crated  thoroughly  t  oeausc  tin-re  arc  not  enough  t  opera¬ 
tion  rn'-a.v.res.  The  high-vt  ir-teivi  re!:::  a  ms  l’ur  meu-utxs  of  similar 
shills  are  ger. orally  found  between  two  printed  tests,  but  printed  te^ta 
rre-a-.wnng  unlike  skills  are  almost  as  closely  related  as  printed  tests 


me. miring  similar  skills. 

There  is  no  evidence  that  instructor  ratings  of  student  proficiency 
are  more  highly  correlated  from  one  performance  check  to  the  next 
than  are  the  cone-ponding  performance  cheek  -cores.  It  had  been 
hoped  that  these  ratings  would  gain  oon.-istency  by  taking  account  of 
the  iriflucr.ee  of  poorly  standarized  chocking  conditions. 

Evidence  seems  to  show  that  there  is  a  considerable  overlap  be¬ 
tween  the  three  basic  skills  of  the  radar  observer.  Insofar  as  printed 
teats  and  performance  checks  are  valid  measures  of  particular  skills, 
navigation  and  bombing  skills  appear  to  be  most  closely  related, 
bombing  and  set  operation  skills  somewhat  les3  so,  and  navigation 
and  r.et  operation  skills  least  related. 

Intcrcorrelations  of  approximately  O.dO  are  found  for  three  sec¬ 
tions  of  Final  Test  I,  each  of  which  includes  relatively  homogeneous 
navigation  computations.  Correlations  of  cadi  of  these  three  sections 
with  a  simulated  mission  section  are  almost  equally  high.  Because  the 
operations  in  the  three  subtests  constitute  a  major  part  of  the  simulated 
mission,  it  was  expected  that  the  correlation  between  the  mission  and 
the  sum  of  the  other  three  sections  would  be  highest  of  all.  Tire  antici¬ 
pated  result  was  found  for  two  samples  of  navigators  but  not  for  a 
sample  of  bombardiers.  A  hypothesis  is  offered  for  this  discrepancy. 

A  comparison  of  proficiency  measure  intcrcorrelations  reported  by 
(lie  Psychological  Research  Project  (Bombardier)  and  the  Psycho¬ 
logical  Research  Project  (Navigator)  indicates  a  general  uniformity 
of  results.  All  three  projects  obtained  their  highest  intcrcorrelations 
between  printed  tests.  None  found  the  correlations  between  printed 
tc..t;>  and  aerial  performance  checks  to  be  high. 
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CHAPTER  NINE 


The  Circular  Error  in  Rad; 
Observer  Trainin'!1 

b 


ir 


INTRODUCTION 

Circular  error  is  a  measure  of  bombin'!  proficiency.  From  (bo 
standpoint  of  being  a  sample  of  the  (ask  required  in  combat,  it  in  tho 
most  direct  evidence  of  accuracy  in  (be  fundamental  operation  toward 
which  radar  training  is  directed.  In  thin  sense,  it.  has  more  inherent 
validity  than  any  other  proficiency  measure.  'Consequently,  it  con¬ 
stitutes  an  obvious  criterion  against  which  to  validate  selection  lentil. 

However,  the  use  of  the  circular  error  ns  a  proficiency  meiisiiro  or 
validation  criterion  is  contingent  upon  its  reliability.  If  u  sludciiL’u 
circular  error  scores  lire  unrelated  to  each  other  or  if  his  standing  in 
his  group  in  regard  to  circular  error  changes  from  one  measurement  to 
tho  next,  ono  must  be  skeptical  of  its  usefulness  for  either  of  tho 
above  purposes.  Therefore,  the  reliability  of  circular  error  is  tho 
central  consideration  of  the  present  chapter.  However,  u  brief  r<  view 
of  recording  and  scoring  procedures  precedes  the  discussion  of  reli¬ 
ability  since  an  understanding  of  these  procedures  is  necessary  to  an 
understanding  of  circular  error  in  radar  training.  The.  reliability  sec¬ 
tion  discusses  the  pertinent  theoretical  considerations  in  determining 
reliability,  and  the  results  of  several  reliability  studies.  Succeeding 
sections  are  concerned  with  other  salient  characters  I ies  of  tbs  circular 
Ci  r  in  training.  One  section  willdi  sen:  i  the  learning  curve  of  radar 
bombing,  and  another,  the  relationships  between  circular  error  ami 
other  proficiency  measures.  A  final  .section  deals  with  an  analyse!  of 
a  systematic  bombing  error  commonly  found  in  radar  oh;  fi  ver  train¬ 
ing,  a  range  error  in  which  bomb  i  tend  to  fall  beyond  the  target. 

RECORDING  ANI)  .MEASURING  ROM  BING  HITS 

During  radar  observer  training,  as  students  practice  direct  and 
coordinated  bombing  procedure-,1  their  bombing  accuracy  for  each 
bomb  run  is  determined  from  records  obtain'  d  by  one  of  line:  UiH.li* 

1  Written  \,J  r.t.  Crv.r".'  S.  Klein. 

1  I' or  a  of  tl. i.romjijrf-i,  r.<-o  rS.ipt»/  1. 
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/ids:  photographs  of  act u:i  1  bomb  drops,  photographs  of  simulated 
bomb  drops,  and  direct  electronic  recordin'/  by  gun-laying  radar 
equipment. 

Photographs  of  Actual  Bomb  Drops 

Most  students  drop  actual  100  pound  bombs  containing  a  small 
powder  charge  on  an  artificial  target  during  one  or  two  training  mis¬ 
sions.  A  terrain  photograph  is  taken  at  the  moment  of  each  impact. 
Since  in  this  type  of  practice  the  impact  point  is  directly  discernible, 
determining  the  bombing  error  from  the  photograph  is  relatively 
simple.  The  scale  of  each  photograph  is  a  function  of  the  altitude 
at  which  it  is  taken  and  accurate  measures  of  error  require,  therefore, 
a  precise  record  of  the  altitude  at  the  moment  of  impact. 

In  general,  radar  bombing  accuracy  is  believed  to  vary  with  bomb¬ 
in':  altitude,  and  circular  errors  arc  corrected  for  this  factor.  The 
usual  practice  is  to  convert  all  circular  errors  to  a  standard  altitude 
of  12,000  feet.  The  intended  effect  of  this  correction  is  to  render  a 
circular  error  found  at  one  altitude  comparable  to  one  obtained  at  any 
other  altitude. 

Photographs  of  Simulated  Bomb  Drops 

The  most  common  type  of  practice  in  radar  training  is 
simulated  bombing,  usually  referred  to  as  “camera  bombing.”  The 
student  performs  all  the  operations  of  bombing  either  direct 
or  coordinated  but  does  not  release  actual  bombs.  A  scries 
of  at  least  three  vertical  photos  of  the  terrain  is  taken  from  the 
aircraft  at  predetermined  points  along  the  fmal  portion  of  the  bomb 
run.  The  exact  time  of  eacli  picturo  is  obtained  either  from  the  time 
settings  on  the  camera  infervulometer,  a  device  permitting  automatic 
photos  at  prescribed  points,  or  from  a  photographic  recording  of  the 
time  as  each  picture  is  snapped.  The  impact  point  in  camera  bombing 
is  a  hypothetical  point  which  must  be  deduced  from  the  photographic 
(lain.  This  is  usually  scored  in  terms  of  circular  error  and  converted 
to  the  standard  12,000  feet  altitude. 

An  important  factor  upon  which  hinges  the  practical  use  of  the 
photo  scoring  techniques  is  the  precision  with  which  the  hypothetical 
impact  points  can  be  located.  Accuracy  checks  on  this  computation 
were  made  by  dropping  actual  bombs  on  an  artificial  target  and  com¬ 
paring  the  bomb  impact  points  with  photo-determined  impact  points 
of  the  same  releases.  It  was  found  that  at  10,000  feel,  under  the  par¬ 
ticular  conditions  of  this  comparison,  impact  points  determined  pho¬ 
tographically  differ  from  actual  impact  points  by  loss  than  100  feet. 

Scoring  is  often  carried  out  with  the  aid  of  bomb  scoring  mosaics. 
In  this  method  the  optical  centers  of  the  several  photographs  taken 
on  the  bomb  rim  are  plotted  on  a  composite  of  several  vertical  photo¬ 
graphs  of  tho  target  area.  The  photographs  are  of  known  scale  and 
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altitude.  From  l lie  plotted  data  on  the  mo~aie,  ohjcctlve  nu  n  ones 
may  be  made  of  track,  ground  speed,  line  heading,  and  altitude.  A 
j i lit*  connecting  the  three  [dotted  [joints  on  the  mo-aic  gives  the  track. 
Ground  speed  is  determined  by  measuring  the  distance  on  the  mosaic 
between  two  of  the  plotted  points  and  relating  it  to  the  time  interval 
between  them.  True  heading  is  found  by  relating  the  longitudinal 
axis  of  the.  photos  to  true  north  of  the  mosaic.  Finally,  photo  altitudo 
may  he  computed  by  relating  the  focal  length  of  the  camera  lens  used 
to  the  distance  measured  on  the  last  photograph  between  any  two 
[joints  a  known  ground  distance  a  part .  With  the  aid  of  the  nbovo 
measures,  and  knowing  the  actual  time  of  fall  of  the  imaginary  bomb, 
the  impact  [joint,  can  be  located  on  the  mosaic. 

The  use  of  the  mosaic  allows  the  scoring  of  larger  bombing  errors 
than  do  the  photographs  alone.  In  instances  where  the  error  is  very 
great,  the  target  may  not  appear  on  a  photo.  In  such  cases  the  mosaic 
is  frequently  large  enough  to  allow  the  plotting  of  impact  point  in 
an  area  of  known  relation  to  the  target.  Errors  so  extreme  as  to  bo 
outside  the  mosaic  area  are  not  scoreable  oven  by  this  method. 

If  the  same  target  area  is  not  frequently  used  for  camera  bombing, 
the  preparation  of  scoring  mosaics  is  not  justified.  Fairly  satisfactory 
.-coring  may  bo  accomplished  without  the  use  of  mosaics.  The  pro¬ 
cedure  is  essentially  similar  to  the  mosaic  method  except  that  the  photo 
data  are  not  [dotted  on  a  mosaic.  The  photos  taken  during  the  bomb 
run  are  put  together  to  form  a  photo  strip.  A  line  connecting  tho 
photo  centers  represents  the  track.  The  point  of  impact,  as  in  tho 
mosaic  method,  may  be  located  with  the  aid  of  true  heading,  track, 
ground  speed,  and  altitude  which  are  obtained  from  the  photo  strip. 
As  mentioned  above,  if  the  bombing  miss  is  so  great  that  the  target  does 
not  appear  on  the  photographs,  the  circular  error  cannot  he  accurately 
scored. 

The  photo  scoring  technique  in  camera  bombing  presents  numerous  ' 
problems  in  its  everyday  application.  It  rarely  provides  a  continuous 
record  of  a  student’s  bombing  performance  in  training  and,  under  somo 
conditions,  its  accuracy  is  limited.  Weather  imposes  n  severe  handi¬ 
cap  in  the  use  of  the  technique.  The  presence  of  an  undereast  prevents 
terrain  photography.  Moreover,  aceurato  photography  requires  that 
the  camera  be  leveled  for  each  picture  and  oriented  with  iv.-pect  to 
the  longitudinal  axis  of  the  aircraft.  Since  camera  equipment  is  not 
gyroslahili/.ed,  undue  turbulence,  will  frequently  upset  the  vertical 
alignment.  In  addition,  deficiencies  in  camera  equipment  or  in  the 
obtained  photographs  often  result  in  indistinct  reference  [joints,  mak¬ 
ing  difficult  the  plotting  of  impact  points.  Inaccuracy  may  al.  o  occur 
from  the  fact  that  the  photo  recording  is  not  wholly  independent  of 
student  mistakes  in  using  the  recording  eqnipnu  nt.  For  example,  the 
“bombs  away”  picture  is  indicated  automatically  by  tho  bomb-sight. 
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If  an  incorrect  disc  speed  has  been  set  into  the  bomb.-ight,  error  will 
result  which  is  unrelated  to  the  slmlont’s  proficiency.  A  major  limita¬ 
tion  of  the  photo  scoring  technique  already  noted  is  that,  frequcntly 
the  bombing  miss  is  so  great  that  the  larg"t  is  not  included  in  the 
picture. 

Recording  by  Cun-Laying  Radar  (SCR-afl t). 

This  method  was  a  late  development  in  the  radar  training  program 
and  never  achieved  wide  use.  It  employs  a  ground  ba-.'d  radar  device, 
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I'niUitK  9.1. — Principal  scores  In  radar  bombing 

developed  originally  for  gunlaying  which  is  placed  in  the  vicinity  of 
the  target  and  trucks  the  aircraft  on  its  approach  to  the  target.  A 
synchronized  recording  pen  provides  a  paper  record  of  the  aircraft’s 
track.  The  aircraft  and  ground  radar  operator  are  in  radio  commitm¬ 
ent  ion.  Through  signals  transmitted  to  the  ground  radar  operator, 
the  release  and  impact  points  can  he  plotted.  A  major  advantage  of 
the  technique  is  that  it  overcomes  tho  weather  difficulties  encountered 
with  photography.  The  possibilities  and  limitations  of  this  method 
are  further  examined  on  page  100  in  connection  with  the  discussion  of 
creating  a  more  reliable  circular  error  measure. 
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n o 1 1 1 1 > i » » Proficiency  Scores 

T!ie  procedures  described  above  yield  throe  principal  types  of  scores 
for  a  bomb  drop.  These  are  shown  in  figure  9.1.  Circular  error  may 
he  defined  as  the  distance  measured  along  the  radius  of  a  circle  whoso 
center  is  the  target  and  whose  circumference  passes  through  tho  bomb 
impact  point.  The  deflection  error  of  the  impact  point  is  the  distanco 
along  a  perpendicular  dropped  from  the  target,  right  or  left,  to  tho 
track  line.  The  range  error  is  the  distance,  over  or  short,  from  tho 
bomb  impart  point  to  the  intersection  of  this  perpendicular  lino  and 
tho  track  line.  In  radar  bombing  training,  tho  latter  two  measures 
were  not  ordinarily  used  sinco  a  more  inclusive  summary  of  bombing 
accuracy  is  provided  by  circular  error. 

Radar  bombing  errors  may  be  analyzed  into  other  components  in 
addition  to  circular  error,  deflection  error,  and  range  error.  Tho 
photo  and  gun-laying  radar  methods  permit  objective  measurement 
of  true  heading,  track,  ground  speed,  and  altitude,  all  of  which  aro 
also  computed  by  the  student  on  each  bomb  run.  Scores  for  theso 
factors  would  add  to  the  variables  measured  for  each  bomb  run. 
Further  study  of  Urge  components  is  warranted  us  part  of  tho  search 
for  reliable  measures  of  bombing  proficiency. 


DESCRIPTION  OF  SAMPLES  USED  IN  CIRCULAR  ERROR 

STUDIES 


Bombing  records  were  selected  for  analysis  largely  from  two  radar 
training  schools,  Boca  Raton  and  Victorville.  Theso  schools  wero 
among  the  first  established  in  tho  radar  observer  training  program 
and  possessed  well-established  bomb  scoring  departments.  As  a  re¬ 
sult,  they  had  available  the  largest  samples  of  pro-VJ-day  bombing 
data.  Records  of  a  third  school,  Langley  Field,  were  used  for  a  singlo 
study,  the  comparison  of  proficiency  measures  and  averago  circular 
error. 

'Fhe  bombing  records  from  the  schools  differ  in  completeness,  num¬ 
ber,  and  scoring  method  used.  All  bombing  at  Boon  Raton  and 
Victorvillo  was  done  "with  the  AN/APQ-13  set  and  was  scored  by 
tho  mosaic  method,  whereas  at  Langley  Field  bombing  wns  performed 
on  AR/ArS-15  or  15 A  and  was  scored  by  the  nonmosaic  method. 
Records  of  actual  bomb  drops  were  available  only  for  Boca  Raton. 
Bocaii  ;o  of  the  differences  existing  among  the  schools,  the  principles 
funding  tho  selection  of  data  from  each  varied  somewhat.  Some 
special  characteristics  of  the  data  for  each  school  may  bo  noted. 


]> 


oca  Union 

All  students  were  rated  bombardiers.  When  the  present  study  be¬ 
au,  data  were  available  for  five  pre-VJ-duy  classes  which  wero 
raduated  prior  to  VJ-day:  555,  G 1 5,  025,  005,  015.  These  classes 
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represented  actual  bombing  records  for  1  to  students  and  camera 
bombing  records  for  M3  students.  "Records  were  selected  for  analysis 
which  had  a  total  of  four  or  more  scored  photographed  drops  for  a 
student,  fn  all,  Ml  actual  bombing  n  cords  and  112  camera  records 
met  this  requirement.  The  minimum  of  four  scored  drops  was  chosen 
because  it  represented  the  best  compromise  between  the.  desirability 
of  bavin"  as  many  drops  as  possible  per  st  udent ,  and  the  undesirability 
of  reducing  the  sample  "ready  by  setting  the  minimum  too  high.  A 
total  of  110  student  records  had  a  minimum  of  four  scored  drops  for 
both  actual  and  camera  bombing.  In  the  actual  bombing  missions 
at  Boca  Ilnton,  virtually  alt  runs  were  made  on  one  target.  The  data 
for  u  second  target  was  discarded  for  the  present  studies. 

Victorville 

In  this  school  the  students  were  rated  navigators.  The  camera 
bombing  records  from  Victorville  were  more  complete  than  those  of' 
either  of  the  other  schools  and  provided  tho  largest  single  sample 
studied.  Individual  records  were  available  for  students  of  19  classes, 
45-16  through  45-35  (except.  45-24),  all  of  whom  had  completed  their 
training  missions  before  VJ-day. 

Only  those  records  which  had  a  total  of  10  or  more  scored  drops 
were  selected  ns  the  sample  used  in  the  reliability  analyses.  This 
selection  provided  a  total  of  400  cases  for  the  odd-even  drop  reliability 
study.  Ten  drops  closely  approximates  the  mean  number  of  scored 
runs  made  by  students  at  this  school.  Uso  of  a  higher  standard  for 
minimum  number  of  drops  would  have  reduced  tho  sample  to  a  much 
smaller  number. 

I.nnglcy  Field 

The  students  at  this  school  were  rated  bombardiers.  Camera  bomb¬ 
ing  records  were  available  for  seven  classes,  45-30  through  45-36, 
totalling  179  cases.  Of  these,  only  1  classes,  comprising  111  students 
bad  completed  all  bombing  training  before  VJ-day.  The  bombing 
records  could  not  be  studied  for  reliability.  Tho  recorded  data  for 
individual  bomb  drops  were  very  meagre  and  no  odd-even  drop  relia¬ 
bility  rncflicient  could  be  computed.  Moreover,  the  number  of  mis¬ 
sions  recorded  for  any  one  student  were  too  few  to  permit  any  study 
of  odd-even  mission  reliability. 

The  limitations  of  these  data  stem  in  part  from  tho  weather  condi¬ 
tions  existing  at  Langley  and  in  part  from  the  use  of  the  nonmosaio 
method  of  scoring.  Weather  was  a  constantly  disturbing  factor,  re¬ 
sulting  in  frequent  changes  in  planned  missions  and  interfering  with 
the  photo  recording  of  drops.  Because  the  mmmosaic  method  was 
us'*d,  extreme  bombing  misses  were  not  scored.  As  was  seen  earlier, 
when  the  target  falls  outside  the  field  of  view  of  the  photographs, 
tho  nonmosaic  method  cannot  estimate  the  circular  error  of  the  bomb- 
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hit  wi<h  :ii vt‘p( able  accuracy.  1  >« mil»in*x  errors  of  this  magnitude 
<  cciirrcd  fnipnmtly  hut  wi'i'o  not  included  in  the  average  circular  error 
i '  in | nil  ed  hy  the  school  for  each  st udeiit.  For  tluve  reasons  the  Lang¬ 
ley  data  proved  useful  only  in  the  study  of  the  relationship  hot  ween 
average-  circular  error  and  proficiency  measures.  The  post  VJ-day 
chives  were  retained  in  order  to  provide  a  fair-sized  sample. 

T!1M  it KLJAIJI L1TY  OF  AVKKAGE  C.IIJCALAK  KKItOH 

A  perfectly  reliable  measuring  device  will  place  individuals  in  tho 
same  rank  order  for  a  number  of  measurements  separated  in  time.  In 
practice,  measurement  with  perfect,  reliability  is  rarely  encountered, 
ft  is  important  to  a  discussion  of  reliability  coefficients  determined  by 
different,  methods  to  note  the  nature  of  the  factors  which  may  influenco 
reliability.  These  factors  fall  into  two  broad  categories:  variation  of 
conditions  under  which  measurement  occurs  and  variations  within  tho 
individuals  measured.  If  students  are  measured  under  different  con¬ 
ditions  from  one  testing  to  another,  such  variations  will,  in  random 
fashion,  penalize  them  (raise  their  circular  error  scores)  at  one  tiino, 
and  help  them  (lowi  r  their  circular  error  scores)  at  another.  Since 
such  results  reflect  variations  in  testing  conditions,  as  well  as  individual 
dilTcronees  in  bombing  accuracy,  the  results  of  a  single  mission  can 
yield  only  limited  generalization  about  true  dilTcronees  in  bombing 
ability.  Likewise,  individuals  do  not  always  perform  with  the  same 
efficiency  from  one  testing  to  another  due  to  variations  in  their  physical 
condition,  motivation,  attitudes,  learning  rate.  etc.  As  in  the  easo  of 
variations  in  testing  conditions,  intraindividuai  variations  will  ran¬ 
domly  raise  or  lower  a  student's  score.  A  reliability  coefficient,  to  bo 
comprehensive,  must  reflect  the  influence  of  both  kinds  of  factors.  It 
must  summarize  the  clfects  of  all  conditions  that  limit  the  extend 
which  generalizations  may  be  made  as  to  true  dilfercnces  in  bombing 
ability. 

A  variety  of  influences  of  the  two  types  mentioned  operate  in  tho 
radar  training  situation.  Some  of  these  arc  shown  in  figure  9.2  in 
terms  of  their  effect  in  altering  a  student’s  rank  from  his  true  rank 
within  the.  group.  By  “true  rank”  is  meant  the  rank  order  determined 
by  a  very  large  (theoretically  infinite)  number  of  bomb  drops  obtained 
lor  an  individual  under  normally  varying  te  ting  conditions.  At  tho 
top  of  figure  9.2  is  an  hypothetical  distribution  of  true  ranks  for  a 
very  large  number  of  radar  observer  students,  each  point  on  tho.  setdo 
re  pre-enting  a  true  rank. 

Bomb  releases  in  radar  training  are  separated  in  time  in  two  ways: 
Within  mission  (from  drop  to  drop)  and  between  mis.-ion  (from  day 
to  day).  Figure  9.2  shows  the  varying  influenco  on  circular  error  of 
inission-to-mission  and  within  mission  (drup-to-drop)  variability  in 
each  of  the  factors  listed.  The  db-  -'hm  of  the  inllucnce  of  each  factor 
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is  shown  by  an  arrow.  The  varying  length  of  the  arrows  is  intended 
to  indicate  that  the  variables  probably  differ  with  respect  to  (heir 
relative  influence  upon  niission-to-mission  variation  and  dmp-to-drop 
variation.  However,  the  relative  importance  of  each  factor  as  shown 
in  the  figure  is  purely  speculative.  More  detailed  consideration  of  the 
specific  sources  of  unreliability  in  radar  performance  and  their  relative 
.significance  will  be  deferred  to  a  later  discussion  (see  p.  1S5).  For  the 
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VARIATION  IN  STUDENT'S 
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A.  VARIATIONS  WITHIN  INDIVIDUAL 

1.  SMALL  PHYSIOLOGICAL  CHANGES,  EG  FATIGUE 

2.  MOTIVATIONAL  CHANGES,'' LET-  OlANGES,  ETC. 

>-  3.  LEARNING-  AMOUNT  FOR  CACH  INDIVIDUAL 
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2. CUAUTY  OF  INSTRUCTION 

3.  PRACTICE  OPPORTUNITIES -SCOPE  TIME 
(MAIN  a  AUXILIARY ), NUV.EER  OF  Q0.V.3  CROPS 


B.  VARIATIONS  IN  TESTING  CONDITIONS 
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Fiuukk  0.2. — Source*  of  mm  liability  of  individual  circular  error  in  radar  bombing 

performance. 


pnseut  it  is  pertinent  to  note  only  that  mission-to-mission  fluctuations 
in  the  variables  listed  influen  ’e  scores  to  a  greater  extent  than  do  drop- 
to-drop  fluctuations. 

Tho  upper  portion  of  figure  9.2  illustrates  how  much  variation 
might  occur  in  an  individual’s  rank  around  his  “truo  rank”  if  all 
ext raindividual  sources  of  unreliability  and  all  effect  of  memory  and 
learning  were  eliminated.  In  such  a  case,  each  successive  circular 
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on  or  measure  would  be  obtained  under  identical  conditions  and 
within  a  learning  plateau  and  would,  consequently,  be  affected  almost 
.-olely  by  small  physiological  and  psychological  changes. 

^valuation  of  Methods  of  Determining  Reliability 
As  pointed  out,  the  merit  of  a  method  for  use  in  estimating  tho 
reliability  of  circular  error  scores  is  determined  by  the  degree  to  which 
it  reflects  the  influence  of  both  mission-to-mission  and  drop-to-drop 
variations.  A  measure  of  reliability  that  fails  to  reflect  mission-to- 


- TECIINICUE  ADCQUATEU  SUMMARIZES  IlitXCATEO  VARIATIONS  Ifl  RAf«  ORDER 

- TECHNIQUE  Cf.’iy  PAR  RALLY  SUMMARIZES  INDICATED  VARIATIONS  IN  RANK  ORDER 

Fioltu  0.3. — Evaluation  of  reliability  methods  applied  to  radar  bombing  per- 
fonnance.  Reliability  coetllelent  should  take  ibto  account  the  variability  of 
the  ranks  of  obtained  circular  error  scores  caused  by  all  the  factors  listed  la 
figure  0.2. 

mission  variations,  in  particular,  seriously  exaggerates  the  extent  to 
which  a  set  of  scores  warrants  generalization  as  to  true  bombing 
.  bilitv.  Figure  9.3  summarizes  a  comparison  of  three  principal 
methods  which  may  be  evaluated  in  this  respect. 

1.  Odd-cvcn  drops  on  one  mission, — Of  the  three  methods,  this  ono 
provides  the  least  adequate  index  of  reliability  since  it  fails  to  measure 
eny  mission-to-mission  variations.  Since  it  is  likely  that  the  effect 
of  v.  it  hi  n  mission  variable  errors  is  relatively  small,  as  suggested  in 
figu ro  9.2,  this  method  gives  a  spuriously  favorable  picture  oi  cir- 
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cuhu*  error  reliability.  An  important  characteristic  of  this  method  ig 
that  factors  which  remain  constant  over  the  entire  mi  -inn  are  cor¬ 
related  with  the  score;,  of  all  drops.  For  example,  lclatiu-ly  tem¬ 
porary  factors  such  as  pilot,  crew,  set,  and  altitude,  which  differ  little 
from  drop  to  drop  within  a  mission,  but  which  would  normally  vary 
over  several  missions,  affect  scores  on  both  halves  of  the  single  mis¬ 
sion  in  the  same  way.  This  spuriously  raises  the  reliability  coeffi¬ 
cient  because,  the  odd  and  even  series  are  affected  equally  and  in  the 
same  direction  by  the  correlated  influences. 

2.  Odd-wen  drop s  on  severed  missions. — The  division  of  drops  in. 
this  method  is  such  that  a  part  of  the  drops  from  each  mission  are 
placed  in  the  odd  series  and  a  part  in  the  even.  As  a  result,  the,  con¬ 
stant  conditions  of  each  mission  will  influence  both  the  odd  and  even 
series  in  the  same  direction,  thus  obscuring  the  attenuating  effect  of 
mis  sic  l-to-mission  variation.  In  this  respect,  the  objection  raised  to 
the  method  of  odd-even  drops  on  one  mission  applies  to  this  method 
as  well.  However,  as  number  of  missions  represen'  »d  in  the  odd 
and  even  series  of  drops  increases,  the  effect  of  conditions  existing 
for  any  one  mission  will  influence  less  and  less  the  size  of  the  odd 
and  even  averages.  Any  interpretation  of  reliability  coefficients  ob¬ 
tained  by  this  method  must,  consequently,  consider  tho  number  of 
missions  from  which  tho  data  are  taken. 

3.  Odd-even  missions. — This  method  provides  the  only  adequate 
estimate  of  reliability  of  averago  circular  error  since  it  takes  into 
account  both  inission-to-mission  and  drop-to-drop  variations.  Since 
the  student’s  performance  is  divided  in  terms  of  missions,  a  correla¬ 
tion  of  the  odd  missions  against  the  even  is  affected  by  all  factors 
which  cause  the  scores  to  fluctuate.  Since  it  measures  the  effects  of 
more  sources  of  variation  than  do  the  other  two  methods,  this  method 
may  he  expected  to  yield  appreciably  lower  reliability  coefficients. 

The  foregoing  comparison  permits  predictions  of  the  relative  sizes 
of  the  reliability  coefficients  likely  to  result  from  application  of  three 
techniques.  In  the  comparison,  of  course,  an  equal  number  of  drops  is 
a  •.unied.  Coefficients  obtained  from  odd-even  drops  on  single  missions 
will  be  higher  than  those  from  odd-even  drops  on  several  missions  and 
both  of  these  will  be  higher  than  coefficients  from  odd-even  missions. 
Coefficients  from  odd-even  drops  on  several  missions  will  decrease  as 
a  function  of  the  number  of  missions  represented  in  the  scries  of 
drops.  As  explained,  the  amount  of  mission-to-mission  variability 
introduced  by  a  large  number  of  missions  should  tend  to  reduce  tho 
effect  of  any  one  extreme  mission  on  both  the  odd  and  even  series  of 
scores,  a  factor  increasing  the  reliability  coefficient  spuriously  in  single 
Jib- don  studies.  Data  to  test  this  latter  hypothesis  were  not  available. 
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■  t.itistiral  Studies  of  Reliability 

'ihe  method  of  odd-even  drops  on  one  mission  could  not  be  used  to 
compute  reliability  in  the  present  study  bc<  au-e  too  few  drops  we ro 
made  within  n  single  radar  training  mission.  Knelt  of  the  two  re¬ 
maining  methods  described  above  was  applied.  In  the  method  of  odd- 
«.ven  drops  on  several  missions,  all  drops  for  each  student  were  divided 
into  an  odd-even  series  and  averages  for  each  series  were  correlated. 
Varying  numbers  of  missions  are  represented  in  the  series.  In  the 
second  method  the  student's  data  were  divided  into  an  odd  and  even 
series  of  missions  and  the  average  circular  error  of  the  odd  series  of 
missions  were  correlated  with  that  of  the  even  series.  In  this  procedure 
the  number  of  drops  within  each  mission  was  permitted  to  vary.  Be¬ 
cause  of  the  nature  of  the  data,  these  methods  were  applied  somewhat 
differently  at  Boca  Baton  and  Victorville.  For  this  reason  the  results 
for  the  two  schools  will  he  discussed  separately.  A  summary  section 
will  bring  together  all  results  and  state  general  conclusions. * 

Boca  Raton. — Although  similar  bombing  procedures  were  used  in 
actual  bombing  and  camera  bombing  practice,  it  seemed  advisable  to 
examine  the  reliability  of  each  form  of  practice  separately.  As  shown 
in  table  9.1  there  is  no  correlation  between  the  two  (>,r=  -.OS,  N==  IIS), 
The  difference  in  average  CE  is  also  considerable  (mean 
actual  — 1,109.82  feet;  mean  camera  =  5, 0I2.S7  feet).  While,  as  will 
be  shown  later,  this  absence  of  correlation  may  be  attributed  partly 
to  low  reliability,  it  is  probably  due  to  some  extent  to  marked  differ¬ 
ences  under  which  the  two  types  of  bombing  were  carried  out.  Actual 
bomb  runs  were  all  on  one  target  and  at  two  altitudes — 10,000  and 
15,000  feet — while  camera  bombing  was  done  on  20  targets  which  va¬ 
ried  in  complexity  and  altitudes  varying  from  8,000  to  20,000  feet.  Tho 
use  of  one  target  simplified  the  navigation  problem  in  actual  bombing, 
'fhe  nature  of  the  targets  differed  sharply  between  the  two  types  of 
practice.  In  actual  bombing  a  pin-point  target  was  used,  surrounded 
by  water  and  easily  recognizable  on  the  scope.  In  camera  bombing, 
aiming  point  recognition  was  a  difficult  problem  sinco  targets  wero 
c  ually  located  in  indnsl  rial  areas  not  easily  different  iable  on  the  scope. 
A  final  consideration  of  unknown  effect  is  that  the  position  of  the  nc- 
t’rd  bombing  scries  in  relation  to  the  camera  bombing  series  varied 
from  student  to  student.  For  some  students,  a<  tnal  bombing  was  done 
early  in  training,  while  for  others,  it  was  done  either  in  the  middle  or 
towards  the  end  of  training.  Comparison  between  the  two  types  of 

1  Tin;  reliability  of  circular  error  may  <l!!Ter  for  direct  anil  coordinated  bo  robin;:.  Sit), 
at  bomldni:  records  at  I>< >t h  Victorville  nml  Hon  Uat<.n  did  not  dNtln;:uh  It  bonildnir  rant 
1 ’ado  1  > v  otic  or  tin*  other  terhnh|ue.  IlowrVcr,  It  U  l;no\\u  ll  at  tlm  naiahcr  of  direct 
1'oiahlni;  raas  attempted  mid  photnyr.i plied  at  the  two  atatha.i  vi'iv  n i- itl I :  1  •  1  \  It  may  bo 
•‘-"mi'd,  therefore,  that  the  reported  reliability  coell’.dent  i  for  actual  and  calm  ra  l.oioMni; 
•"e!!eet  primarily  tho  reliability  of  coo  rill  tint  oil  bomblni;  procedure. 
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practice  will  be  difficult  to  interpret  unless  the  place  of  actual  bombing 
mi  ion  is  held  constant  in  the  entire  series  of  practice  bombing  runs. 


T.uu.k  9.1.—  Correlation  Iclicccn  url'.tnl  bombing  a  ml  camera  bombing  for  US 

Boca  Baton  students1 


Source  o(  uvi  nice  circular  error 

Mean 

SD 

r 

A  rtu. ’ll  1)  lulling  ... 

1.  KKI.  S3 

ffO.  59 

—0. 031 

Culm  rn  bombing . . 

5, 01 Z  *7 

1,773.54 

1  Camern  records  with  3  scort/d  drops  wi  re  admitted  Into  the  sample. 


1.  Odd- even  drops  on  several  missions. — The  mean  number  of  total 
scored  drops  in  bt  th  types  of  bombing  was  considered  to  bo  too  small 
to  hold  constant  the  number  of  drops  in  the  odd  and  even  series  for 
each  student.  As  seen  in  table  9.2,  the  mean  of  actual  bombing  is  8.53, 
with  itn  S.  I).  of  3.15,  and  the  mean  of  camera  bombing  is  5.31  with 
nil  S.  D.  of  2.31.  Only  bombing  records  with  four  or  more  scored 
drops  were  used  in  the  study.  This  provided  reliability  samples  of 
111  and  112  students  for  actual  bombing  and  camera  bombing  respec¬ 
tively.  In  botli  sets  of  data,  average  CE’s  were  computed  for  the  odd 
and  even  series  of  each  student  and  correlated. 


Taiu.k  9.2 — Description  of  Boca  Raton  circular  error  data 


Actual  bombing  data 

Camern  bombing  data 

Vorlnblo 

N 

Menn 

Mcdlun 

SI) 

Rnngo 

N 

Mean 

Me¬ 

dian 

SD 

Range 

Number  of  drops  nhoto- 
fi.iphed  (or  each  stu¬ 
dent  .  . 

145 

X.  53 

a  37 

3.  15 

1 

1  2-17 

143 

5.34 

5.21 

131 

1 1-13 

Number  o (  missions  In 
which  null  student 
nr.ulo  so. red  drop . 

145 

2.20 

2.  71 

.85 

1-5 

143 

3.23 

3.23 

1.00 

HI 

Lowest  nl  1 1 1  lid o  ru 
which  r.'U'h  slmlent 
inndc  :.C'/ii'  1  »hop . 

145 

11,  S 15 

10.0S3 

2,217 

>  10,tXX)  to 

143 

10,070 

10,  373 

1,0SD 

5,000  to 

Jllcbest  nil  Undo  nt 
will'll  cub  student 
in.,  le  i  ini  drop _ 

115 

15,093 

15,453 

1, 3S2 

15,000 

'  10,000  to 

143 

11,231 

15,  2C-3 

2,274 

15,000 

10,000  to 

15,000 

2a  ooo 

'  " 1  'rilmmu  i 'f  (our  drops  rn|iilml  (nr  Inclusion  In  c lit  oven  drop  lellnNIity  stinly. 

1  All  in  In. 1 1  bombing  w  ns  duiia  nt  recorded  nit  illicit,  s  of  (rum  10, (XJO  to  11. (XX)  or  from  15,000  to  10,000  feel. 


The  results  arc  shown  in  lire  upper  portion  of  table  9.3.  The  uncor- 
roctcd  jA  aro  0.-1G  for  actual  bombing  and  0.35  for  camera  bombing, 
both  significant  at  the  1-percent  level.  Corrected  for  length  by  tho 
fipoarmun-Brown  formula,  tho  ?’’s  arc  0,03  for  actual  bombing  and 
0.52  for  camera  bombing.  Since  the  numbers  of  drops  in  tho  odd  and 
even  series  were  not  equalized,  the  Spearman-Brown  formula  is  not 
strictly  applicable.  In  using  it  here,  it  is  assumed  only  that  the  cor¬ 
rected  correlations  approximate  those  which  would  bo  obtained  from 
the  lengthened  series  somewhat  more  closely  than  do  tho  uncorrected 
correlations. 
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[f  learning  occurs  and  if  prael  hv,  a-  i  i  s  <  li  <  • ;  i  ( *  ■  *.  1  by  1 1 1- ■  number  ()f 
scored  drops,  is  permitted  to  vary  from  student  to  student,  tie;  moiv 
tin'  practice  the  simdler  the  average  ('is  of  both  the  odd  and  even  series 
of  a  student.  This  factor  thus  acts  as  a  constant  error,  inMiiencing 
both  series  in  one  direction  and  affecting,  by  itself,  a  positive  cor¬ 
relation.  As  a  means  of  measuring  this  effect  in  the  present  data, 
practice  was  partialled  out  of  the  odd-even  drop  coellic icids.  The 
results  are  presented  in  table  0.4.  It  is  apparent  that  variation  in 
the  number  of  drops  is  a  negligible  influence  in  both  types  of  bombing. 
One  interpretation  of  Ibis  result  is  that  only  a  small  amount,  of  learning 
occurred  in  the  sample  of  bombing  practice  represented  by  the  scored, 
photographed  drops.  All  scored  drops  occurred  on  an  average  of  two 
and  three  missions,  as  seen  in  table  9.2. 


T.mii.k  0.1. —  Odd-cien  drop  reliability  coefficients  with  ami  without  bombing 
practice  parliallcd  out,  Horn  Union  students 


Typi*  of  bombing  practice 

N 

fu  1 

n  i 1 

ru  1 

rn  i 1 

Ad  ml  bombing  . 

m 

-0.  19 

-o  os 

0. -to 

a« 

Cam.  ra  bombing . 

no 

-.09 

-.23 

AS 

.31 

‘  I -odd  distribution;  2**rvcn  distribution;  3«  number  o (  runs  (practice). 


2.  Odd-cvcn  missions. — Only  camera  bombing  data  were  used  in  the 
study  carried  out  by  the  method  of  odd-even  missions.  Actual  bomb¬ 
ing  spread  over  too  few  missions  to  make  possible  a  similar  analysis. 
Since  the  average  number  of  bombing  missions  was  too  small  to  permit 
holding  missions  constant  at  four  (Mean  =  3.23,  SD  =  1.U0),  the  num¬ 
ber  of  missions  was  allowed  to  vary.  However,  only  records  with  a 
minimum  of  two  missions  aro  represented  in  tho  sample.  This  pro¬ 
vided  a  total  of  112  cases. 

The  missions  for  each  student  were  numbered  and  divided  into  an 


odd  and  even  series, 
were,  then  correlated 


The  average  CE’s  of  each  series  of  odd  missions 
'.villi  the  average  Civs  of  the  even  missions.  The 


rang.  >  of  mi.  ions  for  the  odd  series  was  one  to  three  missions,  and  for 
the  own,  one  to  two  missions.  It  is  apparent  from  this  that  the 


systematic  error  due  to  variation  in 


the  number  of  missions  is  not 


li!  i'ly  to  be  groat  enough  to  require  partialling  it  out. 

'Tho  r<  wilts  are  presented  in  table  9.3.  Tho  uncorrected  r  is  0.19, 
significant  at  the  5  percent  level.  When  corrected  for  length  of  the 
vne-i,  this  figure  becomes  0.31.  The  Spearman-Brown  formula  is 
u  ed  with  the  assumption  already  stated. 

VirtoniUc.  1.  Odd-even  drops  on  .several  'missions. — Only  records 
which  had  10  or  more  scored  and  photographed  bomb  drops  were 
selected  as  the  sample  for  all  circular  error  studies  carried  on  with 
the  Victorville  data.  This  provided  n  group  of  400  eases  for  the  study 
of  odd-even  drops  on  several  missions.  For  this  analysis,  the  sample 
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w;is  divided  into  I  wo  groups,  one  roii-i-ling  of  el;i.  es  In  -10  through 
|;,  JiJ  (X  131),  and  (he  other  15-17  through  15  '>5  (X:  'JGS ) .  The 
ditiVrenee  between  the  groups  lay  in  the  treatment  of  runs  recorded  ns 
TN  V-  which  menus  target  not  visible  on  photo.  As  was  seen  earlier, 
urh  drops  wine  dillienlt  to  scon*  accurately  with  the  phot o-.-cnriug 
met  hod.  In  the  ea rly  classes  (  15--1G  through  15-’JO) ,  TN  V  drops  were 
not  scored  hy  the  school.  Either  a  gross  estimate  of  circular  error  was 
recorded  for  these  runs,  for  example,  “greater  than  5.000,"  or  merely 
the  notation  “No  target”  was  recorded.  In  later  classes,  15  -17  through 
■15-35,  the  school  attempted  to  estimate  the  circular  errors  in  such  in¬ 
stances.  In  bombing  records  for  these  classes,  it  is  not  possible,  to 
distinguish  TN V  releases  from  other  bomb  drops.  TNV  drops  rep¬ 
resent  either  extremely  poor  bombing  performance  or  the.  operation  of 
disturbing  influences  beyond  the  student’s  control.  In  the  former 
case,  an  average  CE  which  fails  to  include  such  drops  would  bo 
spuriously  low.  For  this  reason,  an  attempt  was  made  to  take  account 
of  the  TNV  drops  in  the  early  classes  of  the  sample.  Each  such  drop 
occurring  in  the  series  of  10  was  given  an  arbitrary  circular  error  of 
lti, 000  feet.  This  value  closely  approximates  the  highest  circular 
error  found  for  any  single  scored  photo  drop  in  the  Victorville  records. 
The  data  for  the  latter  group  of  classes,  15-21  through  15-35,  could  not 
be  rescored  since  TNV  runs  were  indistinguishable  from  others.  Thus, 
the  study  of  odd-even  drops  for  Victorville  yielded  three  cocflicicnts: 
one  for  each  sample  based  upon  the  uncorrcctcd  data  and  one  for  tho 
first  sample  based  upon  the  data  corrected  for  drops  marked  TNV. 

For  this  study,  practice  was  held  constant.  Only  the  first  10  drops 
for  each  student  were  used  and  those  were  divided  into  an  odd-even 
scries  with  five  drops  in  each  scries.  The  number  of  missions,  on  the 
other  hand,  varied  in  both  the  odd  and  even  series,  hut  did  not  vary 
systematically.  In  other  words,  the  10  drops  represent  a  varying 
mm, her  of  missions  from  student  to  student. 

Since  the  data  for  Victorville  were  more  numerous  and  complete 
than  for  Boca  Raton,  more  detailed  malyses  were  possible.  Odd- 
cven  drop  reliability  was  computed  hy  two  methods:  eorrelat item  of 
•  Id-even  mean  C’K’s  and  correlation  of  odd-even  median  CE’s.  Mean 
(  E  is  average  circular  error  computed  as  the  arithmetic  mean  of  a 
eric*  of  drops;  median  Ola  is  average  circular  error  computed  a:  t he 
mi  lian  of  the  series.  The  latter  method  was  -uggc.tcd  hy  the  hy- 
\  ■  diesis  that  extremely  large  eiivul.ir  errors  include  a  gmater  meas¬ 
ure  of  the  factors  rau.-ing  unreliability  in  the  testing  situation  than 
do  smaller  errors.  Since  such  extreme,  scons  exert  more  elTcct  on  a 
mean  than  on  a  median,  the  median  circular  ei  lor,  if  the  hypothesis  is 
true,  should  reflect  to  a  greater  extent  the  .-Indent’s  tine  bombing 
ability.  This  hypothesis  is,  of  course,  somewhat  oppo.-ed  to  the 
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hypothesis  previou  lv  advanced  that  extreme  bombing  errors  rep- 
re  (.‘lit  very  poor  performance  ami  should  he  included  in  a  student’s 
average  CE  to  reprc. -cut  lii.s  proficiency  adequately. 

The  results  are  given  in  table  All  uncorrected  r\i  between  the 
odd  and  even  means  for  both  early  and  late  classes  aro  fairly  high, 
0.111,  0.0:2.  and  0.09.  All  are  .significant  at  the  1  percent  lewd.  Cor¬ 
rected  by  the  Sp<ei rinan-l’rowii  prophecy  formula,  they  become  0.18, 
0.  IS,  and  0.f>0.  Within  the  early  classes,  the  odd-even  mean  v  when 
TN  V  scores  were  included  is  higher  than  when  they  arc  not  included, 
the  correlation  being  0.0!)  compared  with  0.31.  Jt  is  worth  noting 
again  in  this  connection  that  the  “target  not  visible”  runs  of  a  student 
were  scored  in  their  temporal  position  in  the  series  of  10  drops.  Tho 
arbitrary  scores  assigned  them,  therefore,  occurred  in  unsystematic 
fa  hion  in  the  odd  and  even  series  of  drops.  For  this  reason,  it  is 
unlikely  that  the  rise  in  correlation  was  occasioned  spuriously  by  a 
com lant  error  associated  with  the  TNV  scores.  This  result  suggests 
the  possibility  that  a  more  accurate  method  of  scoring  extremely  high 
circular  errors  will  favorably  affect  reliability.  A  later  discussion 
will  consider  the  point  further. 

The  odd-even  mean  correlations  are  appreciably  higher  than  the 
odd-even  median  correlations  which,  in  the  earlier  and  Inter  samples, 
are  0.27  and  0.1G,  nneorrocted.  It  appears  that  median  CE  will  not  be 
a  more  reliable  score  than  mean  CE.  The  implication  is  that  extremo 
scores  generally  are  not  mainly  due  to  largo  variations  in  external 
conditions  and  are  no  less  typical  of  the  student’s  usual  performance 
than  arc  his  lower  scores. 

2.  Odd  cvcn  missions. — In  this  study  tho  number  of  missions  was 
held  constant  at  four,  the  approximate  mean  number  of  missions  flown 
by  the  group.  Of  the  -100  records  with  10  or  more  drops,  records  were 
selected  which  had  the  required  four  missions.  A  total  of  372  cases 
satisfied  this  requirement.  The  data  for  both  early  and  later  Victor¬ 
ville  classes  were  combined  for  tins  study.  Only  scored  and  photo¬ 
graphed  drops  were  included  in  tho  average  CE  for  each  mission. 
TE  V  drops  of  the  early  classes  were  not  used. 

The  first  four  missions  for  each  student  were  split  into  an  odd  and 
an  even  series.  Average  CE’s  were  obtained  for  the  odd  and  even 
mi  ions  and  the  two  series  were  then  correlated.  Sinco  in  training 
tho  number  of  scored  releases  varied  from  mission  to  mission,  thero 
is  a  po.  Ability  that  tho  odd  and  even  series  were  not  equated  with 
respect  to  practice  However,  this  factor  was  probably  unimportant 
in  tho  present  study  inasmuch  as  a  negligible  relationship  (r—  —0.07, 
N  100)  was  found  between  average  CE  and  number  of  scored  bomb 
runs  (practice)  in  the  Victorville  population. 

Tho  results  are  shown  in  tho  lower  part  of  table  9.3.  The  uncor¬ 
rected  odd-even  mission  reliability  is  0.11,  significant  at  the  5  percent 
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Irvt’l.  CmTirti'il  to  twici’  tin*  length  of  tin'  j)rt‘  cut  series  1))'  menus 
uf  tin.'  Spearman- Brown  formula,  the  r  becomes  0.20. 

Discussion  of  Reliability  Results 

As  predicted  in  the  discission  of  the  three  methods  of  determining 
reliability,  the  odd-even  drop  reliabilities  are  consistently  higher  than 
the  odd-even  mission  reliabilities.  It  was  found  that  the.  reliability 
coell'.ient  of  odd-even  drops  for  actual  bombing  was  highest  of  all. 
The  explanation  suggested  is  that  actual  bombing,  being  concentrated 
into  fewer  missions,  was  less  affected  by  mission-to-niission  sources 
of  unreliability  than  was  camera  bombing.  Such  variation  ns  did 
occur  between  missions  was  less  for  actual  bombing  than  for  camera 
bombing  since,  for  the  former,  bombing  was  carried  out.  on  1  rather 
than  on  '20  targets  and  at  limited  altitudes  rather  then  at  widely  vary¬ 
ing  altitudes.  Another  point  is  that  scoring  actual  bomb  drops  was 
simpler  than  scoring  camera  drops  where  the  impact  point  was  hypo¬ 
thetical  and  computed.  This  would  tend  to  free  the  actual  bombing 
data  from  unreliability  attributable  to  the  scoring  process. 

It  is  to  be  noted  that  even  though  CE  of  actual  bombing  is  more 
reliable,  it  is  not,  because  of  this,  necessarily  a  moro  valid  index 
of  bombing  proficiency.  Since  in  training,  actual  bombs  must  nat¬ 
urally  be  dropped  upon  unrealistic  targets,  the  CE  resulting  may  bo 
less  representative  of  over-all  bombing  ability  than  is  (ho  average  CE 
of  camera  bombing.  It  will  be  recalled  that  no  target  identification 
problems  existed  on  actual  bombing  missions  since  a  single  pin-point 
target  was  bombed.  Of  interest  to  this  comparison  are  the  lower 
correlations  found  between  actual  bombing  CE  and  certain  proficiency 
measures  as  compared  with  the  7-’s  between  camera  CE  and  the  same 
measures.  A  later  section  will  discuss  this  finding. 

Tn  view  of  tho  numerous  influences  which  normally  vary  in  the  radar 
observer  training  program,  the  method  of  odd-even  missions  a  fiords 
the  most  useful  estimate  of  the  reliability  of  circular  error.  Kclia- 
bilities  for  odd-even  missions  arc  low  hut  the  uncorrected  coefficients 
are  significant  at  the  5  percent  level.  The  favorable  olfecton  reliabil¬ 
ity  of  scoring  extreme  bombing  errors  calls  for  improvements  in  the 
procedure  for  recording  bombing  errors  and  computing  average  CE. 

In  another  part  of  this  chapter,  some  of  the  major  :■ oarers  of  unre¬ 
liability  of  bombing  circular  error  in  radar  observer  training  are 
specifically  discussed  and  suggestions  are  made  for  reducing  their  in¬ 
fluence  on  circular  error. 

Comparison  of  Results  With  Studies  of  Visual  Tombing 

Studies  of  reliability  of  average  CE  in  AAF  visual  bombing  are  in 
accord  with  the  general  findings  of  this  study.  Odd-even  drop  ivlia- 
bilities  have  been  reported  to  be  considerably  higher  than  odd-even 
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mix-ion  reliabilities,  with  the  cot  (lineals  for  odd-own  drops  on 
single  mission  being  highest.  Wlnm  sindi  factors  as  pilot.,  bombsiglg, 
aircraft,  and  other  factors  which  normally  vary  in  bombardier  train¬ 
ing  are  permitted  to  remain  uncontrolled  in  the  data,  odd-even  drop 
reliabilities  for  visual  bombing  are  of  the,  same  order  of  magnitude  as 
those  reported  here.  In  general,  however,  the  odd-even  mission  reli- 
abilitics  of  the  present  study,  corrected  /•’ s  of  0.10  and  1,  are  some¬ 
what  higher  than  similar  coefficients  for  visual  bombing. 

In  connection  with  the  somewhat  higher  reliabilil)  of  radar  mission 
bombing,  it  should  be  noted  that  average  CIO  scores  are  considerably 
moro  variable  than  those  of  visual  bombing.  The  standard  deviation 
of  the  radar  scores  are  typically  about  1,500  feet  while  the  .standard 
deviations  of  visual  bombing  CIO  scores  are  approximately  8f>  feet. 
If  the  larger  standard  deviation  reflects  a  greater  variability  among 
students  in  radar  bombing  skill  than  exists  in  visual  bombing  skill, 
the  variance  due  to  uncontrolled  influences  will  be  relatively  loss  for 
the  former.  If  thero  is  a  considerable  range  of  true  ability  within 
the  group,  the  elTect  on  rank  order  of  extraneous  factors  will  bo  less 
than  if  students  differed  from  each  other  by  small  a  uounts. 

Several  visual  bombing  studies  have  succeeded  in  isolating  and 
measuring  the  influence  on  circular  error  of  certain  major  variables. 
Adequate  control  of  theso  factors  results  in  a  marked  increase  in 
reliability.  In  general,  thus  far,  only  statistical  controls  havo  been 
attempted  and  these  havo  had  very  limited  application.  Certain 
variables  snob  as  weather  do  not  easily  lend  themselves  to  control 
through  statistical  adjustment  of  scores.  A. program  of  administra¬ 
tive  control  of  bombing  conditions  in  visual  bombing  training  was 
initiated  in  a  long-range  study  by  Psychological  Research  Project 
( bombardier).  For  a  discussion  of  this  and  other  matters  related 
target  was  bombed.  Of  interest  to  this  comparison  arc  the  lower 
to  the  reliability  of  visual  bombing,  the  reader  is  referred  to  Psycho- 
logical  Pcs.-arch  on  Bombardin'  Training,  report  No.  9  in  the  series 
of  A  AF  aviat  ion  psychology  program  research  reports. 

Variable  Factors  In  Radar  Rombing  .Missions 

The  efforts  made  in  visual  bombing  studies  to  control  nml  correct 
for  the  influence  of  sources  of  unreliability  suggest  n  similar  study  of 
radar  bombing.  This  section  will  explore  some  of  the  major  sources 
of  variation  in  radar  bombing  circular  error  and  suggest  tentative 
methods  for  reducing  their  influence.  These  sources  have  already  beer, 
listed  in  tiguro  9.2.  Some  tentative,  ideas  concerning  their  relative 
influence  may  be  gathered  from  table  9.5  in  which  is  presented  de¬ 
scriptive  data  tabulated  from  the  bombing  records  used  in  the  Victor¬ 
ville  analyses. 
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T.'nu.r.  9.5.-  —  I)f.*eript  ion  of  Virion  illc  circular  error  (lata' 
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Variations  in  bombing  condition*.  1.  Variation *  in  target*. — Tho 
nature  of  targets  bombed  was  varied  differently  at  Langley  Field  tlmn 
at  Bora  Baton  and  Victorville.  In  the  latter  schools  missions,' wore 
planned  in  terms  of  increasing  difficulty  and  complexity  of  targ  "abut 
at  Langley  Field  weather  conditions  and  terrain  resulted  in  the  as¬ 
signment  of  routes  and  targets  in  almost  random,  order.  For  all 
schools  the  relative  difficulty  of  targets  was  unknown  hut  probably 
varied  to  a  considerable  extent. 

Students  varied  al  o  with  respect  to  the  number  of  different  targets 
bombed,  ns  shown  in  table  0.5.  For  (he  data  analyzed  at  Victorville, 
it  mm  pu.  iblo  for  a  stiiduit  to  release  all  bis  bombs  on  as  few  ;ts  two 
t  irgrfs  or  on  as  many  as  10  targets.  The  average  for  all  stmh  nts  was 
seven  targets.  This  fact  is  of  possible  significance  became  of  the 
general  assumption  that  bombing  heroines  more  accurate  a  .  familiar¬ 
ity  with  a  target,  increases. 

‘2.  Variation  in  number  of  mi  ■■■don:;. — Table  0.5  dis.d<  is  that  the 
number  of  mis.  ions  on  which  a  student  dropped  his  minimum  of  10 
bombs  ranged  from  2  to  8.  This  means  that  in  the  odd-even  drop 
r<  liability  study  where  number  of  missions  wa  i  p.  riuilted  to  vary,  the 
records  of  some  students  reflect  to  a  greater  extent  than  others  tho  im:3- 
fnon-to-mksion  variability  in  the  factors  causing  unreliability. 
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3.  Variation  in  bombing  altitude. — At  Victorville,  bombing  altitude 
varied  between  missions  from  7,100  feet  to  18,000  feet,  ns  can  be  seen 
in  tnblo  9.5.  At  Boca  Raton  the  variation  in  camera  bombing  is 
even  greater,  the  range  being  5,000  feet  to  20,000  feet  as  shown  in 
table  9.2.  Similar  data  are  unavailable  for  Langley  Field,  but  it  is 
known  that  variable  weather  conditions  often  necessitated  changes  in 
altitude  ranging  from  8,000  feet  to  20,000  feet.  Table  9.5  discloses 
that  considerable  diirerenc.es  existed  among  students  at  Victorville 
in  the  range  of  altitudes  at  which  each  student  dropped  his  bombs. 
Some  students  dropped  all  their  bombs  within  a  restricted  range  of 
altitudes  during  the  training  period,  while  others  released  theirs  at 
widely  varying  altitudes. 

'I'he  amount  of  fluctuation  in  circular  error  scores  arising  from 
variations  in  altitude  was  not  measured.  As  was  seen  earlier,  radar 
circular  errors  wero  converted  to  a  standard  12,000  feet  altitude.  If 
tho  conversion  factors  aro  correct,  variation  in  altitude  should  not 
lead  to  measurement  error.  This  could  not  be  verified  since  no  in¬ 
formation  was  available  concerning  the  criteria  employed  in  the  con¬ 
struction  of  tho  conversion  table  which  was  in  use  at  the  radar  train¬ 
ing  schools.  In  view  of  the  marked  variations  noted  in  practice 
bombing  altitudes,  this  factor  becomes  of  great  potential  significance. 
It  is  clear  that  the  conversion  factors  used  should  be  determined 
empirically  from  the  altitude  differences  found  in  radnr  training 
rather  than  from  thoso  known  to  exist  in  visual  bombing. 

4.  Variation  in  weather  conditions. — This  factor  was  undoubtedly 
strongest  at  Langloy  Field  where  weather  was  characterized  by  high 
wind"',  turbulence,  frequent  fogs,  and  rain.  Weather  frequently 
changed  during  a  mission  and  this  necessitated  alterations  in  course, 
a  Ultimo  and  target.  A  further  consequence  of  this  typo  of  weather  is 
that  it  severely  limits  the  usoof  the  photo  method  of  scoring.  For  this 
reason,  at  Langley  Field  tho  number  of  usable  photo  drops  for  each 
student  was  quite  small. 

5.  Variation i  in  radar  equipment. — A  highly  important  influence  on 
scores  at  all  schools  resulted  from  variations  in  tho  condition  of  tho 
equipment  from  mission  to  mission  and,  to  some  extent,  from  drop  to 
drop  on  the  same  mission.  Equipment  condition  at  times  ranged  from 
excellent,  opera.! ion  to  complete  malfunction  with  a  single  mission. 

0.  Variations  in  crew  efficiency. — This  factor  varied  considerably 
between  missions  and  to  a  lesser  extent  within  missions.  The  amount 
of  error  attributable  to  crow  efficiency  may  bo  a  function  of  other 
factors,  such  as  time  of  day  of  mission,  first  or  second  halves  of  a 
mission  and  the  experience  and  motivation  of  the  crew.  Examples 
of  crew  operations  in  which  variation  will  influence  CE  are  pilot  per¬ 
formance  in  drift  correction  and  in  maintaining  altitude  or  course, 
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and  bombardier  performance  with  the  bombsight.  Interphone  dif¬ 
ficulties  among  tHe  crew  provides  a  further  illustration. 

7.  Variation s  in  aircraft. — Missions  at  the  three  schools  represented 
in  this  study  were  conducted  in  either  B-17  or  13-21  aircraft.  The 
cramped  quarter,  and  generally  didicult  working  conditions  for  the 
radar  observer  in  the  13-21  as  compared  to  the  B-17  were  a  constant 
source  of  complaint  by  both  students  and  instructors.  Assignment  to 
one  or  the  other  type  of  aircraft  was  not  always  systematic  from 
mission  to  mission.  In  other  schools  where  B-21  and  B-25  aircraft 
were  used,  a  somewhat  different  problem  existed.  The  higher  cruising 
speeds  of  the  smaller  bombers  increased  the  difficulty  of  navigation 
and,  to  some  extent,  shortened  the  time  available  to  the  radar  observer 
on  the  bomb  run. 

Variations  in  the  condition  of  the  aircraft  also  effected  bombing 
scores  to  some  extent.  Thus,  for  example,  the  condition  of  the  inter¬ 
phone  equipment,  the  operating  effectiveness  of  mix'd  illary  instru¬ 
ments,  such  as  the  bombsight  and  altimeter,  and  arrangements  made 
to  increase  the  visibility  of  the  scope  picture  by  blacking  out  the 
working  compartment  could  all  affect  circular  error  to  a  noticeable 
extent. 

8.  Variation  in  briefing  procedure. — Briefing  procedures  at  each 
school,  consisted  of  presentation  of  scope  pictures,  descriptions  of 
routes  and  aiming  points,  and  announcement  of  weather  information. 
These  procedures  varied  in  quality  between  missions  and  between 
briefing  officers.  The  analysis  through  scope  pictures  of  aiming  points 
and  routes  differed  with  respect  to  accuracy  and  completeness.  Some 
briefing  officers  took  pains  to  organize  the  briefing  in  detailed  fashion 
while  others  did  not.  It  is  recognized  that  aiming  point  identifica¬ 
tion  is  a  major  source  of  difficulty  in  radar  bombing.  Variation  in 
the  quality  of  briefing  instructions  could  affect  identification  of  these 
points  and  hence  influence  bombing  scores. 

0.  Variation  in  instructors. — indents  had  instructors  of  varying 
ability  from  one  mission  to  the  next.  Moreover,  the  length  of  training 
missions,  usually  over  4  or  5  hours,  was  known  to  affect  instructor 
motivation  considerably,  depending  particularly  on  the  interval  be¬ 
tween  missions  and  on  whether  a  mission  occurred  in  the  early  morn¬ 


ing,  later  afternoon,  or  at  night.  A  student’s  circular  error  score  on 
any  given  bond)  rim  probably  reflects  in  part  the  qualify  of  instruc¬ 
tion  received  during  the  mission.  Certain  mistakes  in  procedure  can 
be  corrected  by  alert  instruction  after  a  bomb  run  and  this  can  result 
in  lower  circular  errors  on  subsequent  drops. 

Student  variation  in  rate  of  learning. — The  reliability  of  circular 
error  varies  as  a  function  of  the  degree  to  which  learning  rate  varies 
for  different  students.  If  the  rato  of  learning  varies  markedly  tho 
general  tendency  is  for  circular  errors  to  decrease  with  practice  but 
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in  varying  amounts  for  different,  students.  Variations  in  the  amount 
learned  between  bomb  releases  will  reduce  tiie  probability  of  a  stable 
ranking  of  the  student  and  this  is  reflected  in  a  low  reliability 
ooefiicient. 

It  should  be  noted  that  constant  differences  among  individuals  with 
respect  to  practice  or  quality  of  instruction  will  result  in  spuriously 
high  reliabilities.  Differences  in  practice  opportunities  may  take  the 
form  either  of  differing  amounts  of  aerial  training  prior  to  the  first 
bombing  mission,  or  of  differing  numbers  of  practice  missions  after 
bombing  starts.  Tho  former  may  introduce  a  constant  error  by 
determining  a  rank  order  of  individuals  at  the  first  testing  which,  if 
based  upon  largo  differences  in  CE,  will  tend  to  persist  throughout 
training.  The  effect  of  tho  latter  would  be  to  build  up  such  differences 
between  students  as  bombing  practice  proceeds.  Either  of  these  dif¬ 
ferences  will  cause  consistencies  in  rank  order  which  arc  not  attribut¬ 
able  to  differences  in  bombing  ability. 

Tho  Victorville  data  provide  some  evidence  concerning  differences 
among  students  with  respect  to  practice  opportunities.  Table  9.5 
discloses  that  at  tho  time  students  release  their  first  scored  practice 
bomb,  their  previous  experience  with  tho  main  scopo  varied  from  iy2 
hours  to  19  hours  of  practico  for  earlier  classes,  or  from  3  to  27  hours 
for  the  later  classes.  By  tho  timo  a  student  has  made  his  tenth  scored 
release,  his  total  practice  time  can  range  from  11  to  35  hours  for  the 
earlier  classes,  or  from  141/2  to  48Vjj  hours  for  tho  lator  classes.  Be¬ 
tween  his  first  and  tenth  scored  and  photographed  drops,  the  average 
Victorvillo  student  obtained  an  averago  of  about  15  hours  bombing 
practice.  However,  this  varied  among  individual  students  from  4  to 
25  hours. 

It  should  bo  remembered,  of  course,  that  theso  data  do  not  constitute 
continuous  records  of  bombing  practice.  It ol cases  which  could  not  be 
photographed  or  scored  because  of  weather  or  for  other  reasons  were 
not  recorded.  II<  nee,  students  coino  to  their  first  and  tenth  scored 
drops  with  somewhat  more  practico  than  is  actually  shown  in  the 
table.  Thera  is  no  reason,  however,  to  doubt  tho  existence  of  consid¬ 
erable  variability  of  tho  type  described.  Tho  results  shown  in  table 
9.4,  on  tho  other  hand,  indicato  that  differences  in  practice,  as  repre¬ 
sented  by  the  number  of  drops  during  tho  training  period,  had  virtually 
no  influence  on  tho  reliability  coefficients  for  odd-even  drops.  It  ia 
unlikely,  therefore,  that  any  of  tho  correlal  ions  in  tho  various  reliabil¬ 
ity  studies  reported  hero  wero  raised  spuriously  by  this  factor. 

As  against  differences  in  practico,  differences  in  tho  rate  of  learning 
during  training  probably  always  redueo  reliability.  If  tho  amount 
learned  by  different  students  varies  between  successive  measurements, 
the  rank  order  of  tho  students  from  timo  to  time  will  change.  Since 
no  estimato  could  bo  made  of  tho  differential  learning  rates  of  indi- 
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victuals,  the  amount  of  variable  error  of  this  nature  reflected  in  the 
reliability  coellicients  is  unknown. 

In  a  relatively  complex  task  such  as  radar  bombing,  variations  in 
learning  rate  are  probably  greatest  during  the  early  stages  of  learning. 
As  already  noted,  the  circular  error  measures  of  this  study  represent 
the  student’s  earliest  drops.  It  is  possible  the  reliability  of  that  circu¬ 
lar  error  would  bo  loss  affected  if  it  were  obtained  at  a  more  advanced 
j-tage  of  learning  for  the  group,  on  a  terminal  plateau  where  per¬ 
formance  is  stabilized  and  further  improvement  is  not  anticipated. 
As  will  be  seen  in  a  later  section,  evidence  suggests  that  the  average 
student  does  not  reach  such  a  plateau  in  the  curve  of  his  learning  until 
a  considerable  number  of  additional  hours  of  practice  beyond  the 
wartime  training  period. 

Improving  the  Reliability  of  Average  Circular  Error 

Two  types  of  control  are  possible  over  the  sources  of  unreliability: 
statistical  and  administrative.  The  usual  aim  of  the  former  is,  through 
conversion  tables,  to  correct  available  circular  error  data  for  known 
sources  of  variability,  such  as  altitude.  Also,  average  CE  may  bo 
recomputed  to  give  less  weight  to  releases  which  seem  to  reflect 
extraneous  influences  beyond  the  student’s  control;  for  example,  bomb 
releases  of  certain  missions  may  be  eliminated,  medians  rather  than 
means  may  be  used,  and  releases  where  the  photographed  target  was 
not  visible  may  be  resoored.  At  best,  two  difficulties  arise  in  making 
statistical  corrections.  If  the  conditions  arc  not  controlled  under 
which  bombs  are  released,  the  extraneous  influences  aro  likely  to  bo 
complex  and  difficult  to  assess,  creating  a  perplexing  problem  in 
devising  correction  formulae.  The  second  problem  is  that  certain 
factors  such  ns  weather  arc  difficult  if  not  impossible  to  quantify. 

Administrative  control  of  the  sources  of  unreliability  is  concerned 
with  the  conditions  under  which  bombs  arc  dropped  and  the  results 
arc  recorded.  In  this  approach  modifications  are  made  in  the  train¬ 
ing  procedure  so  as  to  oliminato  the  influence  of  known  sources  of 
variation  from  any  single  release.  Certain  pertinent  modifications 
may  be  sugge:  ted  which  might  conceivably  increaso  both  tbo  reliability 
of  the  circular  error  and  its  validity  as  a  measure  of  bombing  ability. 

One  needed  improvement  is  for  recording  and  scoring  procedures 
which  will  provide  a  more  continuous  record  of  bombing  performance 
and  a  larger  number  of  scored  releases.  Reliability  increases  as  a 
function  of  the  number  of  measures  upon  which  it  is  based.  By  in¬ 
creasing  the  number  of  drops  the  influence  of  various  sources  of 
unreliability  on  the  course  of  an  individual’s  performance  may  bo 
more  nearly  randomized. 

Of  the  various  method:;  of  scoring,  scope  photography  and  recording 
by  ground  radar  appear  to  bo  the  two  most  promising  from  the  point 
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of  view  of  increasing  the  proportion  of  recorded  and  scored  releases. 
The  first  method  of  scope  photography  overcomes  certain  limitations 
of  the  present  terrain-photo  method.  In  this  method,  the  impact 
point  is  determined  and  scored  from  scope  photographs;  the  scorer 
merely  takes  a  fix  from  the  best  points  visible  on  the  impact  scope 
photo  and  plots  this  fix  on  a  chart  which  contains  tho  target.  Since 
the  scope  presentations  are  photographed,  weather  is  minimized  as 
a  handicapping  factor  in  scoring  releases.  With  this  procedure  a 
continuous  record  of  each  bomb  run  up  to  the  impact  point  is  recorded. 
The  fact  that  the  scope  camera  is  under  direct  control  of  the  radar 
observer  reduces  the  likelihood  of  camera  malfunctions  or  improper 
camera  settings.  A  further  desirable  feature  is  that  the  area  c  .cred 
by  scope  photographs  is  large,  a  characteristic  permitting  the  ready 
scoring  of  extreme  misses.  A  derivative  benefit  of  this  method  is  that 
it  would  enable  training  authorities  to  insist  upon  a  photographic 
record  of  every  bombing  run  for  each  student.  Hitherto,  on  missions 
where  terrain  photos  of  releases  could  not  be  obtained,  it  is  likely  that 
the  care  taken  in  bombing  practice  was  less  than  might  have  been 
the  case  if  photographic  records  were  possible.  Difficulties  with  this 
j net  hod  are  that  it  requires  trained  experts  to  score  the  picture  and 
detailed  charting  of  local  points  of  tho  target  area  so  that  they  may 
bo  used  in  fix-taking  at  ranges  limited  to  5  or  10  miles.  It  is  possible 
that  scope  interpretation  will  be  easier  in  tho  newer  sets  which  have 
vastly  improved  definition. 

The  second  development  in  recording  devices,  the  ground  radar 
method  described  on  page  170,  also  has  advantages  over  tho  photo¬ 
scoring  procedures  currently  used.  Like  scope  photography,  it  over¬ 
comes  weather  limitations,  obtains  an  accurate  record  of  extreme 
misses,  and  plots  a  continuous  record  of  the  entire  bomb  run.  It  may 
also  involve  less  error  in  determining  impact  point  than  the  photo- 
terra  m  method  although  this  is  ns  yet  undetermined.  Extensive  use 
of  the  method  over  an  entire  mission  route  or  oven*  several  different 
routes  requires  considerable  equipment  inasmuch  as  a  recording  device 
must  be  located  at  every  target.  The  constant  radio  liaison  required 
bet wt  en  the  ground  installation  and  aircraft  also  presents  a  problem. 
Another  cumbersome  feature  is  tho  unwieldy  length  of  tho  record 
for  a  single  bomb  run,  a  characteristic  which  raises  a  considerable  phy¬ 
sical  problem  in  processing  the  detailed  record  sheets  for  scoring.  It 
should  he  noted  that  scores  other  than  circular  error,  range,  and  do¬ 
lled  ion  error  nro  obtainable  with  tho  photographic  and  radar  record¬ 
ing  procedures  just  described.  Photographs  may  be  used  to  score 
objectively  a  student’s  computation  of  heading,  track,  ground  speed, 
altitude,  and  bombsight  sighting  angles.  Such  measures,  if  used, 
would  add  to  the  number  of  variables  scored  for  encli  run.  • 
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The  descriptions  presented  in  a  previous  section  of  the  influence  of 
such  factors  as  weather,  condition  of  equipment,  type  of  aircraft, 
crew,  bombing  altitude,  target,  briefing  instruction  and  instructors 
upon  circular  error  scores  imply  requirements  for  control  which  need 
not  be  enumerated.  Basic  to  clTective  control  of  any  typo,  however, 
is  the  location  of  basic  training  schools  in  geographical  areas  where 
weather  makes  it  possible  to  delimit  the  conditions  under  which  mis¬ 
sions  arc  conducted.  A  point  frequently  made  during  tho  war  was 
that  the  location  of  schools  in  areas  of  relatively  constant  weather 
would  not  provide  practice  under  the  varying  conditions  frequently 
met  in  combat.  However,  during  the  early  stages  of  learning,  constant 
conditions  with  respect  to  weather  and  other  factors  may  facilitate 
rather  than  detract  from  learning.  Radar  performance  under  all 
types  of  weather  conditions  should,  without  question,  bo  a  part  of 
training  at  a  more  advanced  training  stage. 

THE  LEARNING  CURVE  FOR  BOMBING  IN  RADAR  OBSERVER 

TRAINING 

Circular  error  data  are  here  applied  to  two  questions  of  prac¬ 
tical  importance  in  radar  observer  training.  What  is  the  relation  be¬ 
tween  bombing  practice  and  circular  error?  What  is  tho  optimum 
length  of  the  radar  observer  course? 

Relation  Between  Bombing  Practice  and  Circular  Error 

In  approaching  this  question,  tho  assumption  is  made  that  apti¬ 
tude  for  radar  bombing  is  not  associated  with  the  amount  of  practice 
a  student  receives.  In  some  air-force  training  programs  it  was  tho 
policy  to  give  additional  practice  to  poor  students.  Such  a  policy 
would  tend  to  associate  a  large  amount  of  practice  with  high  circular 
error  and  would  lower  the  expected  negative  relationship  between 
circular  error  and  amount  of  practice.  However,  such  a  system  of 
additional  aerial  practice  for  poor  students  was  not  followed  in  radar 
training  and  there  is,  consequently,  no  evidence  that  the  data  of  tho 
present  study  woro  liable  to  a  systematic  bias  of  this  sort. 

For  this  study,  the  index  of  bombing  practice  for  Langley  Field 
and  Victorville  was  flic  total  number  of  bombing  runs,  whether  scored 
or  nnscorcd.  At  these  schools  the  data  included  a  record  of  all  drop3 
cf  a  mission  which  were  unphotographed  or  unscored  for  various 
reasons.  At  Boca  Raton,  the  record  of  tho  number  of  unphoto¬ 
graphed  runs  was  incomplete;  therefore,  only  the  number  of  scored 
and  photographed  runs  was  used  as  a  measure  of  practice.  For  all 
schools,  the  average  CE  based  on  all  scored  and  photographed  drops 
Was  the  measure  of  bombing  proficiency. 
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The  Learning  Curve  for  Radar  Bombing 

Tablo  9.G  gives  the  results  for  the  three  schools.  All  the  corre¬ 
lations  are  low  and  in  the  expected  direction  blit  only  one  is  signifi¬ 
cant  at  the  5-percent  level.  The  single  significant  correlation  sug¬ 
gests  that  the  most  consistent  improvement  with  practice  occurs  in 
actual  bombing.  As  was  seen  previously,  however,  this  typo  of  mis¬ 
sion  was  not  typical  of  the  bombing  situation  generally  encountered 
by  the  radar  observer  and  was  probably  simpler  than  t ne  camera 
missions.  The  time  devoted  to  actual  bombing  was  probably  suffi¬ 
cient  to  show  learning  progress  in  the  simpler  problems  posed.  The 
remaining  correlations  suggest,  on  the  other  hand,  that  the  camera 
missions,  in  which  industrial  targets  were  usually  bombed,  presented 
complex  learning  tasks  with  which  little  progress  can  be  made  on  the 
basis  of  us  little  practice  as  is  represented  by  the  recorded  data. 


T.uii.k  0.0. — Relationxhip  bet  term  bombing  prut:  lice  ax  indicated  by  number  of 

drop x  and  average  circular  error 
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1  At  Horn  Hilton,  total  number  of  scored  photographed  runi  was  taken  at  (he  Index  of  practice  effect. 
On  vs  a  it h  fewer  (him  four  seor  -d  photographed  runs  were  not  Included  because  they  were  not  u'cd  In  the 
nil  ability  study.  This,  therefore,  slightly  curtails  practice  effect  In  (his  sample. 

*  .Significant  nt  5-pcrceut  level. 

The  above  conclusions  are  confirmed  in  learning  curves  based  upon 
five  Boca  Raton  student  classes.  In  figure  9.1  is  plotted  averago  cir¬ 
cular  error  for  the  group  for  both  complex  and  simple  targets.  The 
curve  for  camera  bombing  is  based  upon  113  students,  while  that  for 
actual  bombing  involves  141  students.  For  each  type  of  practico  the 
average  CE  of  the  entire  group  was  found  for  each  consecutive  drop. 
For  some  students,  photographs  either  were  not  taken  of  a  particular 
drop  or  were  not  scorablo.  Consequently,  the  total  number  of  scores 
varies  for  each  drop.  The  number  of  students  entering  the  average 
CE  is  given  on  the  graph  for  each  drop.  A  minimum  of  10  drops  was 
required  before  an  average  CE  was  plotted. 

Inspection  of  figure  9.4  reveals  that  in  bombing  complex  targets 
there  is  a  definite  difference  (approximately  1,000  feet)  between  ac¬ 
curacy  on  the  first  two  drops  and  accuracy  on  later  drops.  Beyond 
this,  however,  little  learning  is  evident.  In  the  curvo  for  bombing 
simple  targets,  learning  occurs  through  the  first  four  drops  (approxi¬ 
mately  700  feet)  but  thereafter  the  curve  quickly  decelerates. 

In  figure  9.5  are  plotted  the  average  circular  errors  of  the  same 
group  of  men  for  consecutive  bombing  missions.  Thus,  all  the  first 
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mission  runs  (if  .-ill  students  are  combined  and  an  average  CE  for  tho 
day  plotted.  This  was  done  for  all  missions  for  which  them  was  a 
minimum  of  15  scored  drops  from  which  to  compute  an  average.  It 
was  found  that  only  scattered  runs  were  recorded  for  a  sixth  mission. 
In  actual  bombing  virtually  all  drops  occur  in  throe  missions.  The 
number  and  percentage  of  drops  used  to  compute  average  CE  for  a 
given  day  are  shown  on  the  graph. 

Figure  9.5  discloses  very  little  improvement  in  the  bombing  of  com¬ 
plex  targets  over  the  period  plotted.  The  decrease  from  the  fourth 
to  the  fifth  mission  must  bo  cautiously  interpreted  because  tho  number 
of  drops  comprising  the  last  plotted  point  is  very  small.  In  actual 
bombing  there  appears  to  bo  slight  but  continual  improvement  from 
the  first  to  the  fourth  mission. 

To  summarize  the  evidence  from  both  sets  of  curves,  the  clearest 
evidence  of  learning  within  the  period  plotted  occurs  on  tho  first  two 
drops  in  complex  target  (camera)  bombing  and  on  the  first  four  drops 
in  simple  target  (actual)  bombing.  The  improvement  consists  of  a 
decrease  in  CE  of  approximately  1,000  feet  in  camera  bombing  and 
about  700  feet  in  actual  bombing.  After  these  early  drops  no  measur¬ 
able  learning  is  apparent. 

However,  these  conclusions  are  subject  to  several  qualifications.  An 
equivocating  factor  in  tho  interpretation  of  tho  learning  curves  is  that 
(ho  circular  error  scores  of  tho  presont  studies  were  not  corrected  for 
target  difficulty.  A  total  of  20  targets  of  undetermined  difficulty  were 
used  at  Boca  Raton.  It  is  unknown  whether  tho  difficulty  of  targets 
was  disregarded  in  assigning  students  to  routes  or  whether  routes  were 
assigned  in  such  a  way  that  targets  became  increasingly  more  difficult 
during  tho  course.  If  target  difficulty  were  handled  in  tho  latter 
fashion,  its  effect  in  tho  present  data  was  probably  to  mask  any  learn¬ 
ing  which  occurred. 

A  second  qualification  relates  to  the  use  of  only  scored,  photo¬ 
graphed  drops.  Scored  photographed  drops  provide  an  unknown 
portion  of  the  total  number  of  drops  made  in  the  entiro  Boca  Raton 
course  and  may  or  may  not  bo  representative  of  all  drops.  Moreover, 
the  true  position  of  scored  drops  in  the  learning  process  is  obscured 
by  the  fact  that  the  exact  sequence  of  scored  drops  in  relation  to  un¬ 
scored  drops  is  unknown. 

The  findings  presented  above  suggest  that  the  radar  observer  train¬ 
ing  program  was  of  insufficient  length  to  develop  a  high  degreo  of  skill 
in  radar  bombing.  Additional  evidence  for  this  conclusion  is  presented 
by  nil  extended  training  experiment  conducted  at  Victorville  by 
project  SO-70,  KS-MG  of  the  National  Defense  Research  Committee.4 
Ono  purposo  or  this  study  was  to  determine  the  amount  of  aerial 

‘"Flnnl  Report  on  Extended  Training  Experiment,"  ner.dquartcra,  Vlctorrlllo  Army  Air 
I'lfld.  Victorville,  Cnllf.,  12  p. 
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triiining  required  for  students  to  achieve  (lie  limits  of  their  proficiency 
in  radar  bombing.  A  group  of  20  men,  10  bombardier  students  selected 
from  Boca  Ha  ton  and  10  navigator  students  selected  from  Victorville, 
were  given  approximately  150  hours  of  aerial  practice  beyond  the  30 
to  35  hours  of  training  of  the  standard  radar  observer  curriculum. 
The  group  was  comprised  of  rnen  of  all  levels  of  ability  as  determined 
by  a  printed  proficiency  test  administered  at  the  completion  of  the 
regular  course.  During  the  experiment,  training  was  conducted  in 
teams.  Each  radar  observer  student  flew  with  the  same  bombardier 
foe  the  duration  of  the  experiment  and,  on  a  large  proportion  of  his 
missions,  was  teamed  with  the  same  pilot  and  copilot.  In  planning 
the  missions,  provision  was  made  for  each  student  to  analyzo  the 
photographs  of  his  previous  bomb  runs.  The  purpose  of  this  pro¬ 
cedure  was  to  acquaint  each  student  thoroughly  with  the  sources  of 
his  own  errors  so  that  he  could  correct  them  on  later  missions. 


TRAINING  HOURS 

Fic.uue  9.0. — NDRC  extended  training  ntmly  learning  curves. 


The  group  results  of  this  study  arc  shown  in  figure  9.0.  The  bomb¬ 
ing  results  of  the  group  were  averaged  for  successive  25-hour  periods 
of  training  and  plotted.  The  average  C'E’s  are  plotted  at  the  mid¬ 
points  of  each  period.  Two  average  CIO’s  are  shown,  one  for  all  runs 
and  one  for  the  first  runs  only.  The  average  CIO  for  all  runs  comprises 
every  scoreablo  run  in  extended  training  made  by  tho  class  and  in¬ 
cludes  many  repeated  runs  over  the  same  target.  In  general,  the  moro 
runs  made  on  a  singlo  target  in  the  same  mission,  the  lov.cr  is  the  cir¬ 
cular  error  for  that  mission.  In  order  to  eliminate  the  influence  of 
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this  repetition  of  runs,  the  CE’s  of  the  first  runs  only  of  ouch  mission 
wore  averaged.  This  average  represents  the  student’s  ability  to  hit 
the  target  on  his  first  attempt.8 

Figure  9.G  reveals  that  approximately  85  hours  beyond  the  usual 
training  period  were  required  for  the  entire  class  “to  reach  such  a 
level  of  proficiency  that  the  circular  error  on  the  first  run  of  each 
mission  was  approximately  equal  to  the  average  for  the  whole  mis¬ 
sion.”0  As  noted  earlier,  we  may  assume  that  the  students  brought 
to  the  experiment  approximately  30  to  35  hours  of  previous  main  scope 
training.  If  this  is  added  to  the  extended  training  time,  figure  9.G 
may  he  translated  to  mean  that  the  steepest  part  of  the  learning  curves 
comes  at  about  50  hours  of  training  (20  hours  extended  training), 
that  learning  begins  to  level  oil  at  about  05  hours  of  training  (35  hours 
extended  training),  and  does  not  reach  its  terminal  plateau  or  peak 
until  about  115  hours  of  training  (85  hours  of  extended  training). 
It  must  be  noted  again,  however,  that  the  extended  training  experi¬ 
ment  differed  from  the  basic  training  situation  in  at  least  two  import¬ 
ant  respects.  Improved  opportunities  were  provided  for  analysis  of 
one’s  own  bombing  errors,  and  bombing  was  carried  out  by  relatively 
constant  crew  combinations  which  made  it  probable  that  teams  rather 
than  individuals  woro  learning.  It  may  be  expected  that  a  greater 
number  of  training  hours  would  bo  required  to  reach  the  same  degree 
of  accuracy  under  customary  training  conditions.  The  conclusion 
assumes,  of  course,  that  othor  aspects  of  training  are  unchanged  and 
that  only  tho  amount  of  aerial  practice  is  varying.  The  exorbitant 
prico  in  aerial  training  time  poses  the  question  of  how  other  features 
of  the  course,  c.  g.,  ground  school  classes  and  trainers,  may  be  improved 
in  order  to  introduce  economy  in  tho  number  of  hours  required  fer  a 
student  to  reach  peak  efficiency. 

Figures  9.5  and  9.G  invite  comparison  because  of  tho  fact  that  the 
learning  curves  of  figure  9.6  (extended  training  experiment)  begin 
approximately  at  a  point  in  training  where  tho  day-to-day  curve  of 
figure  9.5  ends,  i.  e.,  at  about  30  hours  of  training.  The  sharp  improve¬ 
ment  at  <ho  early  part  of  tho  curve  for  the  extended  training  experi¬ 
ment  is  in  striking  contrast  to  tho  plateau  seen  in  figure  9.5  for  the 
preceding  period  of  training.  As  a  possible  explanation  for  this 
difference,  attention  is  again  called  to  tho  features  of  tho  extended 
training  experiment  which  favor  improvement  in  CE,  namely,  team 
learning,  and  self-analysis  of  bombing  errors.  Had  these  conditions 
existed  in  regular  training,  it  is  possiblo  that  tho  two  sets  of  curves 
might  havo  been  more  continuous. 

»  A  que.-dlon  arisen  In  thin  connection  regarding  tl>c  effect  of  Increasing  familiarity  with 
tnrgctn  In  llie  training  area  upon  apparent  Improvement  In  bombing  accuracy.  For  a  more 
ilolnllnl  description  of  thin  feature  of  Urn  experiment,  the  reader' Is  referred  to  "Final 
Report  on  listen  tied  Training  Experiment,”  op.  olt. 

«  "Final  Report  on  Extended  Training  Experiment,"  op,  clt.,  p.  5. 
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N;i  vigator  students  were  found  in  the  extended  t  raining  stud  v  to  be 
<•, insistent I)'  superior  to  bombardier  students  with  re.speet  both  to 
It  liming  progress  and  peak  proficiency.  'The  navigators  reached  near 


peak  proficiency  at  only  do  hours  of  additional  practice  while  bombard¬ 
iers  required  approximately  So  hours  of  additional  aerial  time. 
Moreover,  the.  difference  between  the  two  persisted  even  after  plateaus 
laid  been  reached.  The  average  CE  of  navigators  at  peak  performance 
was  l,oSo  feet;  that  of  bombardiers  was  t.fiTS  feet.  The  average  CE 
for  the  entire  group  was  1,C>27  feet.  It  should  be  meat  ioned  that  initial 
differences  in  favor  of  the  navigator  students  might  have  resulted  from 
the  fact  that  all  had  been  previously  trained  at  Victorville  where  the 
study  took  place,  whereas  the  bombardiers  had  all  been  trained  at  Boca 
Baton  and  were  unfamiliar  with  the  Victorville  routes  and  targets. 


RELATIONSHIP  BETWEEN  AVERAGE  CE  AND  OTHER  PRO¬ 
FICIENCY  MEASURES 

A  matter  of  some  interest  from  the  point  of  view  of  the  interrelation¬ 
ship  of  different  proficiency  measures  is  the  degree  to  which  CE  is 
related  to  various  test  and  courso  grades.  All  such  relationships  for 
which  data  were  available  were  analyzed  and  are  reported  in  this 
section.  In  the  first  and  second  studies,  an  analysis  is  made  of  the 
relationship  between  CE  and  course  grade,  with  attention  called  to  the 
effect  of  including  extreme  misses  ill  CE  and  partialling  out  the  effects 
of  practice  on  CE.  In  the  third  and  fourth  studies,  CE  is  correlated 
with  proficiency  measures  presenting  the  radar  observer’s  specific  skills 
rather  than  his  over-all  proficiency.  Throughout  this  section  and  in 
tallies  0.7, 0.8, 9.9,  and  9.10  the  signs  of  correlation  coefficients  have  been 
reversed  so  that  positive  correlations  indicate  the  association  of  “good” 
performance  in  the  two  variables. 

Relation  Between  Final  Course  Grade  and  Average  CE,  Com¬ 
puted  With  and  Without  Extreme  Misses 

The  correlations  computed  in  analyzing  the  relationship  between 
final  course  grade  and  CE  when  the  latter  is  computed  with  and  with¬ 
out  the  extreme  misses  represented  by  TN  V  drops,  are  based  upon  data 
from  Langley  Field  camera  bombing  records.  A  total  of  179  students 
were  available)  for  Langley  Field  classes  lo-ffO  through  -lTi-ffO.  Of 
there  seven  classes,  four  bad  entirely  completed  their  bombing  prior  to 
VJ-day.  For  171  of  the  179  students,  it  was  possible  to  obtain  the  filial 
course  grade  assigned  by  the  training  station.  A  description  of  this 
course  grade  is  given  in  chapter  11. 

The  Langley  Field  bombing  records,  partly  as  a  result  of  tho  non- 
moraie  method  of  photo  scoring  used  at  that  station,  contained  an 
unusual  number  of  TNV  drops.  In  order  to  examine  the  effect  of  those 
extreme  bombing  errors  on  the  correlation  between  CE  and  other 
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proficiency  measures,  two  compulations  of  average  CE’s  were  carried 
out.  One  of  these  was  based  solely  on  scored  photo  drops.  The  other 
included,  in  addition,  the  TNV  drops  each  of  which  were  assigned  a 
circular  error  score  of  8,000  feet.  This  figure  was  chosen  as  being 
somewhat  higher  than  the  highest  scored  circular  error  for  this  sample. 

The  effect  of  including  the  TNV  drops  upon  the  correlation  with 
final  course  grade  is  shown  in  table  0.7  which  gives  the  coefficients  based 
upon  both  nncorrected  and  corrected  average  CE’s.  The  inclusion  of 
the  TNV  drops,  as  shown  in  the  coefficient  with  the  corrected  average 
CE,  has  the  effect  of  raising  the  correlation  with  course  grade  from  0.07 
to  0.15,  an  increase  of  doubtful  statistical  significance. 


Tam.e  0.7. — Relationship  between  av crape  circular  error  and  course  grade  for 

17.'i  students,  Langley  Field 
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i  Significant  at  the  5-percent  level. 


Relation  Between  Final  Course  Grade  and  CE,  With  and  Without 
Bombing  Prnctice  Purtinllcd  Out 

The  analysis  of  the  relationship  between  final  course  grade  and  CE, 
with  and  without  bombing  practice  partialled  out,  wero  based  on  data 
obtained  from  three  training  stations,  Boca  Baton,  Langley  Fiold,  and 
Victorville.  Tho  Langley  Field  data  is  described  in  the  preceding  sec¬ 
tion.  The  data  from  Boca  Raton  and  Victorville  are  those  analyzed  ear¬ 
lier  in  the  chapter  in  connection  with  the  investigation  of  CE  reliabil¬ 
ity.  The  correlations  for  all  three  schools  are  presented  in  table  9.  8. 
Since  the  difference  among  students  with  respect  to  bombing  experience 
could  conceivably  affect  both  the  average  CE  and  final  course  grades, 
it  appeared  advisable  to  partial  out  this  factor.  For  Boca  Raton,  tho 
index  of  bombing  practice  was  tho  total  number  of  scored,  photo¬ 
graph  al  drops  (insufficient  information  was  available  concerning  un- 
pliotographed  drops) ;  for  Victorvillo  and  Langley  Field  it  was  tho 
total  number  of  releases,  including  unphotographed  as  well  as  photo¬ 
graphed  drops. 

It  is  seen  from  table  9.8  that  practice  effect  has  no  influence  on  any 
of  the  relationships.  Combining  tho  camera  bombing  results  for  the 
three  schools  by  Fisher’s  Z-method,  the  correlation  between  average 
CE  nnd  final  course  grade  is  0.10  (N  — 070),  which  is  significant  at 
the  1-percent  level.7  The  combined  r  of  0.10  is  based  on  uncorrected 

•Tin*  assumption  la  modi1  In  combining  these  results  that  the  threo  samples  aro  Independ¬ 
ent  random  autnplea  from  the  same  general  population  ot  radar  observer  atudeuts. 
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average  C’K’s.  It  may  l.“  concluded  tluil  :i  low  hut  stable  relation 
exists  between  the  final  proficiency  grades  of  each  school  and  bombing' 
>kill  as  measured  by  average  circular  error.  'The  higher  a  student’s 
course  grade,  the  lower  the  average  CE  lie  is  likely  to  have. 


T.MH.K  !V8.- — Correlations  behreen  average  circular  error  ini'!  final  course  grade 
ici/h  (Did-  icilhout  bombing  practice  part iallcd  out 
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It  is  interesting  to  note  in  table  9.8  that  camera  bombing  appears  to- 
be  more  closely  related  to  proficiency  as  measured  by  final  course  grado 
than  is  actual  bombing.  This  difference  is  consistent  with  a  point 
made  earlier  that  actual  bombing  on  a  pin-point  target  which  is  easily 
identifiable  involves  fewer  of  the  skills  taught  in  the  course  than  does 
camera  bombing  where  industrial  areas  provide  very  complex  targets. 

Relation  Between  Phase  Grades  and  Circular  F.rror 
In  this  analysis  the  average  CE  for  Boca  Raton  mid  Langley  Field 
sf ’’dents  were  correlated  with  phase  grades  computed  by  the  Psycho¬ 
logical  Research  Project  (Radar)  and  based  upon  standardized  pro¬ 
ficiency  measures.  Three  such  grades  were  computed;  a  lliglit  stanine 
based  upon  aerial  performance  check  scores,  a  trainer  stanine  ba-ed 
upon  bench  set  and  supersonic  performance  check  scores,  and  a  class¬ 
room  stanine  based  upon  proficiency  fest  scores.  These  tests  and  per¬ 
formance  checks  are.  described  in  chapters  f>  and  0.  The  computation 
of  the  three  phase  grade  stainiues  is  described  in  chapter  11. 

The  Boca  Raton  students  are  drawn  from  the  sample  used  in  the 
analyses  of  CE  reliability.  The  Langley  Field  students  are  those  de¬ 
scribed  in  the  two  studies  just  represented.  In  computing  CE  for 
the  Langley  Field  students,  unscored  extreme  misses  were  included. 
Of  tht'  179  Langley  Field  students  for  whom  bombing  records  were 
available,  phase  grades  could  he  obtained  for  only  97. 

'Fable  9.9  shows  the  correlations  obtained.  'Flic  correlation  of  CE 
with  course,  grades,  based  upon  standardized  proficiency  niea-nres,  i3 
included  for  comparative  purposes.  The  coefficients  in  the  table  sug¬ 
gest  again  that  actual  bombing  CE  is  less  related  to  other  proficiency 
measures  than  is  camera  bombing  CE.  For  (lie  course  grade  stanine„ 
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tho  Hi -Ilf  stanine,  :md  (lie  t  miner  stanine  the  combined  Boca  Raton  and 
Langley  Field  samples  provide  statistically  significant  correlations. 
The  eorrelntion  with  course  grade  stanine  is  significant  at  the  1-per¬ 
cent  level,  while  those  with  (light,  and  trainer  stanincs  are  significant 
at  the  5  percent  level.  The  correlation  with  tho  classroom  stanine 
does  not  reach  statistical  significance. 


Taiii.k  n.O.  Correlations  of  average  circular  cnor  with  course  grades  and  pro¬ 
ficiency  stanincs  computed  by  the  radar  project  for  Lanylcy  Field  and  Boca 
Raton  students 
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Relation  Between  Single  Proficiency  Measures  and  CE 

Tho  phase  grade  stanincs  described  above  were  not  available  for 
Victorville  students.  However,  scores  for  this  group  could  be  obtained 
on  specific  standardized  proficiency  tests  and  performance  checks. 
Four  such  measures  were  correlated  with  average  CE:  they  are  the 
jFinnl  Aerial  Performance  Check,  Radar  Final  Test  I  and  II,  Radar 
Navigation  Intermediate  Test,  and  Radar  Bombing  Intermediate 
Test.  The  number  of  students  for  whom  scores  could  bo  obtained 
varied  for  the  different  measures. 

Tho  correlations  obtained  aro  shown  in  table  9.10.  With  tho  ex- 


Taut.k  0.10. — Correlations  of  average  circular  error  from  camera  bombing  with 
standardized  proficiency  measures  for  Victorville  students 
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eeption  of  Part  II,  Radar  Final  Proficiency  Test,  the  correlations  of 
all  tests  with  CE  attained  statistical  significance  at  or  ubovo  the  5- 
percent  level.  Tho  coefficients  for  tho  different  tests  are  approximately 
equal  in  size.  •  That  obtained  for  the  aerial  performance  check  is  also 
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of  the  panic*  order  of  magnitude,  but  since  it  is  based  upon  a  smaller  N, 
its  stability  is  uncertain. 

SYSTEMATIC  ERROR  IN  RADAR  BOMBING  TRAINING:  THE 

RANGE  OVER  ERROR 

Ordinarily,  one  would  expect  that  among  experienced  radar  ob¬ 
servers  the  directional  errors  in  bombing  would  be  unsystematic  over 
a  period  of  time,  that  errors  would  occur  as  frequently  in  one  direction 
as  in  another.  In  basic  radar  observer  training  this  was  not  the  case. 
Table  9.11  gives  the  average  amount  and  direction  of  the  range  error 
of  camera  bombing  at  each  of  the  three  schools.  The  direction  of  tlio 
error  is  consistently  over  or  beyond  the  target.  The  data  arc  taken 
from  the  population  samples  used  in  tho  other  circular  error  studies 
reported  in  this  chapter.  Results  comparable  to  those  in  table  9.11 
were  found  for  the  first  25  hours  of  extended  training  in  tho  XDRC 
extended  training  experiment.  For  TIG  rims  during  this  period  nn 
average  range  over  error  of  1,074  feet  is  reported.  In  addition  to 
this  data,  108  bombing  runs  scored  at  Langley  Field  by  tho  radar 
method  (SCR  584)  yielded  an  over  error  of  1,420  feet.  This  finding 
minimizes  the  possibility  that  the  directional  error  is  n  function 
of  the  photo-terrain  method  of  scoring. 


Taiu.k  0.11. — Constant  range  ciror  for  camera  bombing  In  radar  observer  training 
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While  the  possible  causes  for  this  error  aro  clearly  not  limited  to 
misuse  of  his  equipment  by  tho  radar  observer,  an  exploratory  analysis 
of  these  causes  was  undertaken.  It  was  planned  that  this  analysis  bo 
followed  by  experiments  which  might  reveal  tho  responsible  factors. 
Although  training  was  terminated  before  such  experiments  could  bo 
conducted,  the  hypotheses  developed  are  of  sufficient  interest  to  war¬ 
rant  presentation.  Discussion  is  divided  into  causes  arising  from 
equipment  sources  and  those  arising  from  psychological  sources. 

Equipment  Sources 

2'rail  error  in  ; photo-scoring  of  direct  bombing  runs. — Trail  is  tho 
distance  behind  the  aircraft  at  which  a  given  bomb  will  strike.  Tho 
bomb  lags  behind  the  aircraft  because  nir  resistance  tends  to  ovcrcomo 
its  forward  speed,  as  shown  in  figure  9.7.  In  making  nil  actual  bomb 
rolcaso,  compensation  is  made  for  this  lag.  In  tho  calibration  of  tho 
radar  computer  drum  ground  speed  lines,  which  are  used  in  direct 
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bombing  for  determining  the  correct  bomb  release  circle,  this  compen¬ 
sation  is  accomplished  by  inclusion  of  an  average  trail  value. 

In  figure  9.7  it  is  seen  that  the  line  AB  represents  the  axis  of  the  ver¬ 
tical  camera  at  the  time  of  the  impact  picture;  henco  point  B  will  be 
the  center  of  the  picture.  The  camera  takes  no  account  of  trail  since 
it  photographs  the  impact  point  or  the  point  directly  under  the  air¬ 
craft.  But  since,  as  noted  above,  the  radar  computer  is  calibrated  in 
such  a  way  as  to  allow'  for  trail,  the  aircraft  will  have  traveled  beyond 
the  aiming  point  at  the  time  of  impact.'  and  the  photograph  taken  at 
(his  point  will  be  in  error  in  the  over  direction.  It  becomes  apparent 
that  the  impact  point  of  any  simulated  bomb  release  in  direct  bombing 


Fkjcui;  9.7. — Diagrammatic  sketch  of  aircraft  anil  bomb  at  instant  of  impact. 


will  be  plotted  trail  distance  too  far  over,  unless  this  valuo  can  also  be 
taken  into  account  in  scoring  tho  impact  photograph. 

It  has  not  been  possiblo  to  determine  the  exact  trail  valuo  included 
in  the  computer,  but  it  is  known  to  be  approximately  CO  mils  at  12,400 
feet,  tho  averago  bombing  altitude  at  Langley  Field.  Scored  as  zero 
trail,  GO  mils  of  trail  would  amount  to  an  over  error  of  744  feet.  The 
smaller  range  error  obtained  at  Victorvillo  (see  tablo  9.12  below) 
may  possibly  be  accounted  for  by  tho  fact  that  no  direct  bombing  was 
dono  at  that  school. 

Trail  error  in  coordinated  bombing . — In  contrast  to  direct  bombing, 
tho  bomb  releases  of  coordinated  bombing  practice  were  not  corrected 
for  trail  so  as  to  innlco  the  impact  point  photograph  correct.  This 
required  that  the  trail  arm  of  thobombsight  (index  for  setting  propor 
trail  valuo  in  mils)  bo  set  at  zero.  If  a  student  neglected  to  do  this 
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iiud  either  sot  trail  or  left  trail  in  the  bombsight,  an  over  error  would 
result  on  tho  impact  point  photograph  at  an  average  rate  of  12.4  feet 
per  mil  of  trail.  A  similar  error  was  that  of  preset  trail.  Preset  trail 
is  an  internal  misadjnstment  in  the  bombsight  resulting  in  an  actual 
trail  value  differing  from  the  reading  set  by  the  trail  arm.  If  preset 
trail  is  present,  it  is  usually  positive,  ft  may  be  that  students  were 
not  consistently  briefed  to  prelligbt  tin:  bombsight  for  this  malfunction. 

Failure  to  -use  sector  scan. — The  time  between  renewals  by  tho 
sweep  of  tlie  target  return  on  the  scope  may  be  a  factor  contributing 
to  error.  On  normal  rotation  the  aiming  point  is  ronewed  every  2.5 
seconds.  Assuming  a  ground  speed  of  204  miles  per  hour,  the  aiming 
point  would  be  approximately  750  feet  nearer  the  center  of  the  scopo 
the  next  time  the  sweep  passed.  With  the  30°  sector  scan  tho  target 
return  is  renewed  every  0.83  second  with  AX/APQ-13  and  every 
0.12  second  with  AN/APS-15  and  AN/APS-15A.  With  30°  sector 
scan  it  would  be  only  120  or  249  feet  nearer,  depending  upon  the  equip¬ 
ment  used.  Sinco  most  radar  observer  students  would  not  rolcaso 
bombs  before  they  saw  the  aiming  point  touch  the  bomb  relcaso  circle, 
failure  to  use.  sector  scan  could  result  in  over  error.  Tho  movement 
of  the  target  on  a  full  revolution  of  a  360°  scan  is  illustrated  in 
figure  9.8. 

Psychological  Sources 

Uncertainty  delay. — When  a  student  loses  or  is  uncertain  of  his 
aiming  point  ho  may  have  to  take  too  long  for  definite  identification. 
At  a  ground  speed  of  204  miles  per  hour  the  effect  of  uncertainty 
delay  upon  direct  bombing  range  error  and  displacement  error  in 
coordinated  bombing  is  to  create  an  over  error  at  the  rnto  of  about 
20°  feet  per  second.  In  coordinated  bombing,  a  delayed  final  sighting 
angle  signal  will  set  up  a  slow  rate  in  tho  bombsight  which  also  results 
in  an  over  error. 

Target  difjlculty. — Probably  uncertainty  delay  increases  ns  a  func¬ 
tion  of  target  difficulty.  In  this  case,  the  more  difficult  tho  target,  tho 
larger  the  o\cr  error.  If  it  were  possible  to  obtain  a  suitable  criterion 
of  target  difficulty  a  test  of  this  hypothesis  could  be  made. 

Delay  in  starling  the  camera  intcrvalonu'ter. — In  either  direct  or 
coordinated  bombing,  delay  in  starting  the  camera  intervalomotcr 
should  account  for  very  little  error  sinco  only  simple  reaction  time  m 
involved.  However,  in  many  trailing  aircraft,  the  intervnlomctcr 
was  out  of  the  bombardier’s  reach  and  some  delay  in  stniting  it 
resulted. 

Failure  to  use  leading  edge  of  target  and  leading  edge  of  lomb 
circle. — It  was  frequently  reported  at  radar  observer  training  stations 
that  aerial  incdructors  were  advising  procedures  other  than  the  nso 
of  iho  coincidence  of  tho  leading  edges  of  bombing  cirelo  and  aiming 


point  for  sighting  bombs  and  calling  sighting  angles  to  the  bombardier. 
Moreover,  there  was  a  tendency  for  bombardier  students  to  alter  the 
procedure  by  waiting  until  the  bombing  circle  was  over  the  center  of 
the  aiming  point.  Because  of  special,  characteristics  of  the  radar 
equipment,  maximum  accuracy  can  be  achieved  only  when  the  stand¬ 
ard  procedure  is  used.  Using  the  middle  of  the  bombing  circle  will 


Fjouke  O.8.— -Apparent  movement  of  the  target  during  one  full  revolution  of 

the  sweep 

result  in  an  over  error  of  250  feet  (since  tho  width  of  tho  circle  on 
bombing  rangi  represents  a  ground  distance  of  500  feet). 

Synchronization  lag. — In  a  coordinated  bomb  run  tho  bombardier  is 
required  to  synchronize  the  rate  mechanism  of  the  bombsight  (which 
in  part  controls  the  range  of  the  bomb  to  be  dropped)  with  tho  speed 
of  approach  to  the  target,  ns  given,  by  tho  radar  operator  who  calls 
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‘•mark” 
icr  Joe  ' 
a  given 


as  the  t nrget  reaches  successive  sight  ing  angles.  The  bombard- 
this  b}'  adjusting  two  knobs:  when  tlu;  radar  operator  calls 
angle,  the  bombardier  moves  the  displacement  knob  so  ns  to 


place  the  sighting  angle  index  at  that,  angle,  and  also  the  rate  knob 
to  correct  the  acceleration  rate  of  the  sighting  angle  index. 

Because  of  the  series  of  perceptual  debus  and  the  reaction  time  in¬ 
volved  in  the  process  of  synchronization  (particularly  at  the  final  sight¬ 
ing  angle),  it  is  apparent  that,  while  in  most  cases  an  approximately 
correct  rate  may  be  established,  the  successive  displacement  settings 
will  be  behind  the  proper  value.  At  an  altitude  of  12,-100  feet,  this 
results  in  an  error  of  approximately  117  feet  per  degree  of  lag. 


Controlling  the  Range  Over  Error 

It  is  believed  that  the  sources  of  error  listed  in  the  foregoing  section, 
particularly  those  involving  scope  interpretation,  are  readily  sus¬ 
ceptible  to  elimination  or  reduction  by  additional  instruction  and 
practice.  This  conclusion  is  borne  out  in  the  results  of  the  Extended 
Training  Experiment  conducted  by  NDEC  which  are  presented  in 
table  9.12. 

Table  9.12  discloses  that  the  rango  error  decreases  progressively 
up  to  100  hours  of  training.  Also,  after  25  hours  of  extended  training, 
the  directional  error  is  as  likely  to  ho  short  ns  it  is  over.  Two  factors 
may  account  for  the  decrease  in  over  error  after  the  first  25  horn’s  of 
extended  training:  (a)  The  beginning  student  exhibits  deficiencies  in 
set  operation,  navigation  and  bombing  procedure,  or  scope  interpreta¬ 
tion,  which  cause  over  error  and  arc  eliminated  by  further  practice 
and  instruction;  (&)  students  were  allowed  to  analyze  their  bombing 
errors  over  a  considerable  period  of  time,  thus  becoming  increasingly 
aware  of  the  systematic  nature  of  their  range  error.  This  would  aid 
them  in  developing  compensating  techniques  which  would  tend  to 
place  the  point  of  impact  nearer  the  aiming  point. 


T.\ju.b  9.12. — Size  and  direction  of  range  error  for  auccc.uivc  25-hour  periods 

of  extended  training  * 
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SUMMARY 

In  this  chapter  are  discussed  five  topics  related  to  bombing  errors 
in  radar  observer  training:  methods  of  lecordiug  and  scoring  bombing 
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circular  error,  the  reliability  of  circular  error,  the  learning  curve 
for  circular  error,  the  relationship  between  circular  error  am!  trainin'* 
proficiency  measures,  and  the  systematic  over  error  in  range. 

Recording  and  scoring  procedures  were  adapted  to  score  two  typos 
of  bombing  practice:  simulated  or  camera  bombing,  the  most  frequent 
type,  and  actual  bombing.  The  most  commonly  used  recording  and 
scoring  system  in  the  wartime  training  program  was  photo  bomb 
scoring.  In  this  technique,  actual  bomb  drops  are  scored  by  taking 
a  to.  rain  photograph  at  the  moment  of  bomb  impact.  To  score  simu¬ 
lated  bomb  drops,  a  series  of  vortical  terrain  photos  arc  taken  during 
the  bomb  run.  From  these  photos  the  impact  point  of  the  simulated 
release  is  plotted  and  the  amount  of  error  scored.  Scoring  is  facili¬ 
tated  by  the  use  of  a  mosaic  photograph  which,  since  it  encompasses  a 
wider  area  than  any  single  photograph,  allows  the  measurement  of 
larger  errors. 

The  principal  limitations  of  photo-terrain  scoring  arc  that  weather 
severely  restricts  its  use,  that  turbulence  will  frequently  upset  the 
vertical  alignment  of  cameras  which  arc  not  gyro-stabilized  and  that 
extreme  bombing  misses  cannot  bo  scored.  These  limitations  result 
in  ail  incomplete  record  of  the  student’s  bombing  performance  and 
contribute  to  unreliability  of  the  circular  error  measure. 

Simulated  bomb  runs  may  bo  recorded  also  by  means  of  a  gun-laying 
radar  device.  This  method,  which  was  developed  near  the  end  of 
wartime  training,  not  only  overcomes  the  weather  handicap  of  terrain 
photography,  but  also  makes  it  possible  to  score  extreme  misses. 
However,  it  presents  difficult ies  in  the  amounts  of  equipment  required, 
tho  radio  liaison  required  between  the  aircraft  and  the  ground  radar 
installation,  and  the  unwieldy  length  of  the  record  for  each  bomb  run. 

Bomb  releases  are  usually  scored  in  terms  of  circular  error,  which 
is  converted  to  a  standard  12,000  feet  altitude.  However,  additional 
components  of  tho  bombing  run,  such  as  heading,  track,  ground  speed, 
and  altitude,  may  he  objectively  scored.  Measures  such  ns  these  are 
as  yet  unexplored  ns  indices  of  bombing  performance. 

An  analysis  was  made  of  the  reliability  of  circular  error  in  order 
to  evaluate  its  usefulness  as  a  validation  criterion.  .  Circular  error 
scores  in  training  are  a  (Tee  ted  by  numerous  influences  unrelated  to 
bombing  ability.  After  considering  tho  problem  of  the  differing  re¬ 
sults  yielded  by  several  methods  of  computing  reliability,  it  was 
pointed  out  that  the  most  useful  reliability  coefficient  was  one  which 
summarized  the  effect  on  circular  error  scores  of  all  factors  that  cause 
a  student’s  rank  to  vary  from  one  day  to  another.  Three  methods  of 
determining  reliability  were  evaluated  according  to  this  standard  :  the 
correlat  ion  o  f  odd  and  even  drops  over  a  single  mission,  the  correlation 
of  odd  and  even  drops  over  several  missions,  and  tho  correlation  of 
di  •ops  from  odd  with  those  from  even  missions.  It  was  concluded 
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tluit  the  first  two  methods  exaggerate  the  reliability  of  circular 
iiror  because  they  fail  in  whole  or  in  part  to  measure  tho  influence 
of  mission  to  mission  variations.  It  was  further  concluded  that  tho 
odd-even  missions  method  provides  the  most  accurate  estimato  of 
reliability  since  it  fakes  into  account  all  of  tho  known  mission-lo¬ 
rn  issi  on  as  well  as  drop-to-drop  influences. 

Application  of  tho  method  of  odd-even  drops  over  several  missions 
to  available  data  for  tho  Boca  Baton  school  yielded  coeTicients  of 
0,10  for  actual  bombing  (N  =  144)  and  0.35  for  camera  bombing 
(X-110).  'When  corrected  for  length  by  the  Spearman-Brown 
prophecy  formula,  these  coefficients  rose  to  0.G3  and  0.52,  respectively. 
Because  practice  (number  of  scored  drops)  was  permitted  to  vary  in 
the  Boca  Baton  sample,  its  influence  was  parlialled  from  the  coeffi¬ 
cients,  and  was  found  to  be  negligible  in  both  actunl  and  camera 
bombing.  The  higher  odd-oven  drop  reliability  found  for  actual 
bombing  was  attributed  to  several  causes,  among  which  are  its  con¬ 
centration  on  a  few  missions,  the  simple  target  used,  and  relative 
objectivity  of  scoring. 

On  data  available  for  the  Victorville  training  station,  odd-oven 
drop  reliability  was  determined  by  two  methods:  (a)  correlation  of 
mean  CE’s  of  odd  and  even  drops,  and  (b)  correlation  of  median 
OK’s  of  odd  and  even  drops.  The  latter  method  supposes  that  largo 
CE’s  reflect  irrelevant  factors  in  the  testing  situation  and  that  these 
factors  unduly  afTect  mean  CE.  Consequently,  median  CE,  which  is 
less  affected,  may  reflect  to  a  greater  extent  tho  bombing  ability  of  the 
student.  Uncorrected  r' s  obtained  with  tho  method  employing  means 
were  0.31  and  0.32  for  two  samples  of  131  and  2GS  cases,  respectively, 
as  compared  with  0.27  and  0.1G  for  tho  same  samples  with  tho  odd- 
even  median  method.  This  result  suggests  that  extreme  scores  gen¬ 
erally  are  no  less  indicators  of  true  bombing  ability  than  arc  the  lower 
circular  error  scores,  and  appeal's  also  to  emphnsizo  tho  importance 
of  scoring  accurately  extreme  bombing  errors  and  including  them  in 
the  average  CE. 

Extreme  bombing  misses,  ordinarily  not  scorcablo  by  tho  plioto- 
ferrain  technique,  were  given  an  arbitrary  score  in  one  group  of  the 
Victorville  sample.  Inclusion  of  these  drops  bad  a  favorablo  effect  on 
odd-oven  mean  drop  reliability,  the  uncorrectcd  coefficient  for  this 
sample  being  0.30  (17-"- 131)  as  compared  to  an  r  of  0.31  found  when 
Bin  extreme  scores  were  not  included. 

Odd-oven  mission  reliability  was  studied  in  two  samples.  For 
Boca  Baton  data,  from  camera  bombing  only,  the  obtained  coefficient 
"‘as  0.10  (>7  —  122).  This  is  increased  to  0.31  when  corrected  by  tho 
Spearman-Brown  formula.  Odd-even  mission  reliability  for  the 
Victorville  data  was  found  to  be.  0.11  uncorrectcd  (N~372)  and  0.20 
’"‘hen  corrected.  From  the  odd-even  i  iis.-ion  cociT.cVnts  obtained  at 
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tho  two  stations,  it  may  he  concluded  that  the  reliability  of  circular 
error  in  radar  observer  training  is  probably  between  0.20  and  0.30. 
This  reliability  is  thought  to  be  suflieient  to  encourage  further  work 
to  improve  circular  error  as  a  criterion  measure. 

The  odd-even  mission  coellicients  obtained  in  this  study  appear  to 
be  somewhat  higher  than  those  found  in  comparable  studies  of  visual 
bombing.  A  possible  explanation  of  this  difference  is  that  there  is 
less  variability  among  students  in  visual  bombing  proficiency  than  in 
radar  bombing.  Consequently,  the  differences  between  students  in 
visual  bombing  are  more  easily  obscured  by  variations  in  factors  un¬ 
related  to  bombing  ability. 

An  exploratory  attempt  was  made  in  this  chapter  to  describe  cer¬ 
tain  sources  of  unreliability  of  circular  error  and  to  suggest  menus 
for  reducing  their  influence.  Among  the  variations  in  bombing  con¬ 
ditions  which  probably  affected  scores  were  variations  in  (a)  target 
difficulty  and  familiarity,  (b)  number  of  missions  over  which  drops 
are  spread,  ( c )  bombing  altitude,  ( d )  weather  conditions,  (e)  radar 
equipment,  (/)  crew  efficiency,  (y)  working  conditions  and  auxiliary 
instruments  in  aircraft,  (A)  briefing  procedure,  and  (i)  instructor. 
The  effect  of  variations  in  practice  and  in  learning  rate  was  also  dis¬ 
cussed.  Iliular  scope  photography  and  recording  by  ground  radar 
appear  to  be  the  two  scoring  developments  which  offer  most  promise 
of  increasing  reliability. 

In  a  study  of  the  relationship  between  number  of  bombing  runs  and 
average  CE  at  three  radar  schools,  correlations  near  zero  were  ob¬ 
tained.  This  suggestion  that  the  amount  of  improvement  in  circular 
error  with  practice  was  negligible,  was  supported  by  an  analysis  of 
circular  error  learning  curves.  Group  learning  curves  based  on  five 
classes  at  Boca  Raton  disclosed  that  improvement  occurred  only  on 
the  first  two  camera  drops  (decrease  of  approximately  1,000  feet  in 
circular  error)  and  on  the  first  four  actual  bombing  drops  (decrease  of 
approximately  700  feet  in  circular  error).  After  these  early  drops,  the 
evidence  tor  future  learning  was  ambiguous.  On  actual  bombing  mis¬ 
sions  there  appeared  to  be  son  improvement  from  tho  first  to  tho 
fourth  mission,  but  camera  bombing  failed  to  show  a  similar  trend. 
If  it  is  assumed  that  the  plotted  data  were  representative  of  progress 
made  during  tho  training  period,  it  may  bo  concluded  that  30  to  35 
hours  of  main  scope  aerial  training  is  insufficient  to  yield  much  improve¬ 
ment  in  bombing  accuracy. 

That  30  to  35  hours  is  inadequate  to  develop  highly  skilled  radar 
observers  is  supported  by  the  results  of  an  extended  training  study 
of  bombing  progress  conducted  by  NDRC.  In  this  study  tho  20  par¬ 
ticipating  students  were  provided  an  opportunity  to  analyze  their  own 
bombing  errors  and  were  teamed  with  other  crew  members  for  tho 
i  lajor  portion  of  the  experiment.  The  study  revealed  that,  under 
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t]u. -t.  comlit i<*ns  of  training.  the  steepest  part  of  the  iojirnin*r  curve 
f  ,|  tin-  group  cauie  at  about  50  hours  of  aerial  training  (‘JO  hours  of 
t’\! eihlctl  I  raining) ,  that  learning  began  to  level  olT  at  about  (lb  hours 
of  practice  (05  hours  of  extended  training),  and  reached  its  terminal 
pint  ran  or  peak  at  about  1  lb  hours  of  training  (8b  hours  of  extended 
training).  It  is  likely  that  team  training  and  self-analysis  of  bomb¬ 
ing  errors,  facilitate  learning.  Siueo  these  were  conditions  special 
to  the  experiment,  it  is  probable  that  the  number  of  aerial  hours  re¬ 
quired  to  reach  equivalent  proficiency  would  be.  greater  under  war¬ 
time  training  conditions. 

Studies  were  made  of  the  extent  to  which  bombing  performance 
may  he  predicted  from  classroom,  trainer,  and  (light  grades  at  three 
radar  schools.  A  low  but  stable  relation  was  found  to  exist  between 
the  final  proficiency  grades  of  each  school  and  average  CE  for  camera 
bombing.  The  combined  r  for  three  schools  was  0.1(1,  with  a  total  N 
of  1170.  Actual  bombing  was  found  to  be  virtually  unrelated  to  final 
course  grade,  a  finding  consistent  with  the  fact  that  actual  bombing 
involved  fewer  of  the  skills  taught  in  the  course  than  did  camera 
bombing. 

Average  CE  was  also  correlated  with  proficiency  measures  pre¬ 
pared  by  the  Radar  Project.  The  relationship  between  camera  bomb¬ 
ing  CE  and  a  final  course  stanine  based  on  the  entire  battery  of  tests 
ami  cheeks  was  low  but  significant;  the  r  for  t he  combined  Boca  Raton 
and  Langley  Field  samples  was  0.20,  N  =  178.  For  a  stanine  based 
upon  aerial  performance  checks  and  one  based  upon  trainer  perform¬ 
ance  checks,  the  correlation  with  CE  was  somewhat  higher  than  for  a 
stanine  based  upon  several  printed  proficiency  tests.  The  F s  between 
actual  bombing  CE  and  these  measures  are  insignificant.  In  studies 
of  the  correlation  between  CE  and  individual  tests,  coefficients  of  the 
order  of  magnitude  of  0.15  were  obtained.  A  single  performance 
check  furnished  a  similar  result. 

In  the  earlier  stages  of  radar  observer  training,  the  range  error 
is  systematic  and  in  the  over  direction.  Probably  the  major  psycho- 
h'gioa.l  factor  contributing  to  this  directional  error  is  an  uncertainty 
T'biy,  winch  is  the  result  of  poor  scope  interpretation,  target  diffi¬ 
culty,  and  perhaps  inefficient  set  operation.  Secondary  psychological 
lectors  apprear  to  be  (a)  failure  to  use  the  leading  edges  of  both 
<er  .vt  end  bombing  circle,  (5)  synchronization  lag,  and  (o)  delay  in 
sf  rting  tho  camera  iutervalometer.  It  is  likely  that  these  errors 
li  e  with  practice  and  instruction. 


History  of  Radar  Observer 
Selection  ’ 


A  history  of  the  select  ion  of  radar  observers  involves  four  psycho¬ 
logical  organizations:  Project  SC-70,  NS-MG  of  tho  National  Defenso 
Research  Committee,  which  is  referred  to  hereafter  Us  tho  NDIiC 
Project;  tiro  AAF  Aircrew  Evaluation  ami  Research  Detachment 
No.  1,  referred  to  as  AE11D  No.  1;  the  Psychological  Research  Proj¬ 
ect  (Navigator),  referred  to  as  the  Navigator  Project;  and  Head¬ 
quarters,  AAF  Training  Command.  This  chapter  reviews  tho  con¬ 
tributions  of  each  of  these  organizations,  points  out  their  interrela¬ 
tions  and  briefly  summarizes  tho  data  presented.  Some  of  tho  re¬ 
search  described  deals  with  operators  of  air-to-snrfacc-vossel  radar 
equipment,  referred  to  ns  ASV.  The  work  of  ASY  operators  is 
roughly  comparable  to  that  of  radar  observers  and  has  been  included 
primarily  because  of  its  importance  to  later  developments  in  rndnr- 
observer  research. 


SELECTION  RESEARCH  BY  PROJECT  SC-70,  NS-146, 
NATIONAL  DEFENSE  RESEARCH  COMMITTEE 


The  selection  of  radar  observers  originated  with  a  research  con¬ 
tract  recommended  to  tho  National  Defence  Research  Committeo  by 
the  National  Research  Council.  The  NDRC  project  established  for 
titis  work  began  its  investigations  at  Camp  Murphy,  Fla.  in  Febru¬ 
ary  1013,  Later,  it  moved  to  Boca  Raton  Army  Air  Field,  Fla.  Tho 
facilities  of  tho  project  were  made  available  to  both  tho  Army  and 
Navy,  and  emphasis  in  research  was  placed  upon  specific  problems 
recommended  to  it  by  one  or  the  other  of  these  services. 

One  of  the  initial  undertakings  of  tho  NDRC  project  was  the  con¬ 
struction  of  tests  for  relection  of  ASV  radar  operators.  Thro  oper¬ 
ators  ns'-d  the  SCR717A  and  SCR717B  radar  sets,  which  were  similar 
to  those  later  used  by  radar  observers.  A  job  analysis  revealed  a 
number  of  specific  tasks  involved  in  satisfactory  ASV  operation,  such 
ns  Iho  “ability  to  detect  pips  on  an  oscilloscope  screen,  to  c  limate  ivla- 
>>ve  pip  heights,  to  set  accurately  a  pip  to  a  hairline  or  gate,  to  convert 
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information  from  one  form  to  another  quie.dy,  and  to  read  and  inter¬ 
pret  accurately  certain  .symbols  and  relationships  involved  on  two- 
dimensional  plots.”5  It  was  concluded  that  the  measurement  of 
these  abilities  would  require  tests  which  would  determine  the  potential 
radar  operators  speed  and  accuracy  of  perceptual  discrimination, 
his  alertness  and  persistence,  and  his  capacity  for  making  quick 
judgments. 

Tests 

On  the  basis  of  the  results  of  this  job  analysis,  along  with  related 
studies  and  observations  of  the  radar  operator  and  his  equipment, 
the  NDRC  project  prepared  13  printed  tests  and  administered  them 
experimentally  at  various  operating  stations  under  the  AAF  Tactical 
O  iter,  Orlando,  Fla.,  at  Camp  Murphy,  Drew  Field,  and  at  the  Naval 
Training  School,  Virginia  Beach,  Va.  A  brief  description  of  each 
of  the  tests  follows:  * 

Form  Detection  Teat. — This  tost  is  designed  to  measure  speed 
nnd  accuracy  in  matching  identical  forms.  It  consists  of  42  lines 
of  12  irregular  forms,  only  2  of  which  are  the  same  size  and  shape. 
Tlio  task  is  to  select  from  each  line  the  two  forms  that  are  identical. 
Time  limit,  8  minutes. 

Form  Conversion  Speed  Teat  I. — Twelve  different  irregular  forms 
arc  presented,  coded  to  represent  a  letter  of  the  alphabet  from  A 
through  L.  The  forms  aro  grouped  to  constitute  short  words  which 
must  be  decoded  by  the  use  of  the  figure-letter  code  key.  Time  limit, 
3  minutes. 

Form  Conversion  Speed  Test  II. — This  is  the  same  as  Form  Con¬ 
version  Speed  Test  I,  except  that  Arabic  numerals  from  zero  to  nine 
nro  substituted  for  letters  of  tho  alphabet,  nnd  a  different  set  of  sym¬ 
bols  are  used.  Time  limit,  3  minutes. 

Scale  Reading  'Test. — In  each  of  4S  items  there  is  presented  a  me¬ 
ter  with  an  indicator  set  at  some  point  on  t he  meter  scale.  Tho 
task  is  to  determine  tho  exact  scale  reading.  A  description  of  a 
machine-scored  version  of  this  test,  Scale  Reading,  CPG37A,  is  in¬ 
cluded  in  appendix  A.  Time  limit,  18  minutes. 

Oscilloscope  Reading  Test. — This  test  contains  40  sets  of  G  circles. 
In  each  circle  there  is  a  pattern  of  vertical  lines  of  differing  heights 
drawn  on  a  horizontal  base  line.  The  task  is  to  determine,  in  each  set 
of  six,  which  2  patterns  are  exactly  alike.  Time  limit,  10  minutes. 

Spot  Location  Test, — This  test  consists  of  one  large  circle  blocked 
off  into  lettered  segments,  and  nine  small  circles  each  containing 

1  Selection  an  1  triUnlnj,*  of  osellloerope  opcm lorn,  Mrniornnihim  No.  3,  Instruction  Man- 
uni  for  Oaclllo  cope  Operator  Touts,  Experimental  Kill  lion,  N'DUC  l’rojcct  SC-70,  NS-H9, 
Camp  Murphy,  l'la. 

•Fur  n  complete  description  see:  Selection  nnd  training  of  oscilloscope  operators  Memo¬ 
randa  No".  'J  and  3.  limtnmtlon  Manual  for  Osellluiuope  Operator  Testa,  Experimental 
Editions,  .N1MIC  Project  SC  70.  N'S-HO,  Camp  Murphy,  Fla. 
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ill  dots.  Tin.*  t  :i >lv  is  to  dctei  mine  into  which  lettered  segment 
0f  the  circle  each  of  the  dots  in  the  small  circle  would 

fall  if  the  l.-uter  circle  were  as  large  ns  the.  former.  A  description  of 
:v  machine-scored  version  of  this  test,  Spot  Location  Test,  CP818A, 
is  included  in  appendix  A.  Time  limit,  1  minutes. 

Course  Location  Test. — In  this  test  there  lire  10  pairs  of  circles. 
One  circle  of  each  pair  contains  n  pattern  of  connected  lines  which  is 
to  bo  duplicated  in  the  other  circle.  The  latter  circle  contains  only 
dots  upon  some  of  which  the  duplicated  pattern  must  full.  Time  limit: 
5  minutes. 

Ratio  Estimation  Test. — Ninety-six  pairs  of  two  vertical  lines  aro 
presented,  each  pair  in  a  rectangular  box.  The  task  is  to  determine 
the  ratio  of  the  left  hand  line  to  the  right  hand  lino.  Tho  lines  nro 
spaced  to  simulate  double  pipping  of  some  radar  sets.  A  description 
of  a  machine-scored  version  of  this  test,  Ratio  Estimation  Test, 
CP225A,  is  included  in  appendix  A.  Time  limit :  5  minutes. 

Target  Course  Analysis  Test. — This  test  consists  of  fivo  groups 
of  small  circles,  each  group  arranged  to  form  a  rectangle.  Tho  circles 
at  the  top  are  numbered  from  1  to  10.  From  each  of  the  top  circles  a 
line  originates,  criss-crosses  the  rectangular  field  in  random  pattern, 
and  ends  at  one  of  the  blank  circles  along  the  bottom  or  sides  of  tho 
rectangle.  The  task  is  to  follow  by  eye  the  line  from  the  numbered 
circle  sit  the  top  to  its  end  at  one  of  the  side  or  bottom  circles,  and  then' 
to  place  in  the  end  circle  the  number  of  the  circle  from  which  the  lino 
originated.  Time  limit:  15  minutes. 

Plot  Reading  Test. — A  large  square  is  blocked  off  into  25  equal 
parts  by  the  uso  of  perpendicular  lines.  Each  side  of  tho  squaro  is 
marked  of!  equally  into  50  numbered  units.  The  task  is  to  locate  each 
of  50  points  found  within  the  square  by  giving  the  numbers  for  the  pair 
of  lines  that  describe  its  location.  Time  limit :  10  minutes. 

Coordinate  Plotting  Test . — Around  the  outer  edge  of  a  largo  circle, 
degrees  are  marked  oil  in  intervals  of  five.  From  the  center  of  tho 
circle  to  the  outer  edge,  nine  concentric  circles  lire  drawn,  with  the 
distance  between  each  circle  representing  5  miles.  Fifty  pail’s  of 
numbers  are  presented,  one  number  indicating  degrees  nrmind  tho 
circle,  the  other,  miles  from  the  center  of  the  circle.  The  task  is  to 
locate  the  point-  in  the  circle  which  is  described  by  the  pair  of  numbers, 
by  placing  a  heavy  dot  at  the  correct  spot.  Time  limit:  10  minutes. 

Poordivatc  Reading  Test. — This  is  the  reverse  of  the  Coordinate 
Riot  ting  T,  st.  Points  are  already  marked  inside  the  circle,  and 
the  task  is  to  locate  the  point  by  giving  a  pair  of  numbers,  one  repre¬ 
senting  degrees  around  the  circle,  the  other,  miles  from  the  center  of 
the  circle.  A  description  of  a  machine-scored  version  of  this  test. 
Coordinate  Reading  Test,  CP221A,  B,  is  included  ^n  appendix  A. 
Pine  limit:  15  minutes. 
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Polar  Grid  Coordinate  Test. — The  sealed  cmle  dimr'ilx  1  in  t lio 
two  preceding  tests  is  presented  with  grid  coordinate  lines  superim¬ 
posed  upon  it.  The  test  involves  two  tasks.  The  first  is  the  same  as 
the  Coordinate  Plotting  Tost:  i.  e.,  locating  a  point  within  a  circle 
described  by  a  pair  of  numbers.  The  second  task  is  to  determine  the 
grid  coordinates  of  this  point.  A  description  of  a  machine-scored 
version  of  this  test,  Polar  C i rid  Coordinate  Test,  CPM'JB,  in  included 
in  appendix  A.  Time  limit:  IS  minutes. 

During  the  summer  of  1013,  three  additional  tests  were  constructed. 
These  were:  Air-borne  Oscilloscope  Reading  Test,  Oscilloscope  Con¬ 
version  Test,  and  Oscilloscope  Interpretation  Test.  ‘‘The  first  two  of 
tho  above  tests  arc  concerned  with  air-borne  radar  operator  functions, 
especially  the  interpretation  and  transmission  of  symbolic  directions 
from  operator  to  pilot.  The  Oscilloscope  Interpretation  Test  is  a 
test  of  ability  to  detect  different  types  of  signals  through  varying  de¬ 
grees  of  background  masking  or  jamming  and  is  applicable  to  all  typos 
of  radar  presentations.” 4  A  description  of  a  machine-scored  version 
of  this  test,  Oscilloscope  Interpretation,  CP817A,  is  included  in  ap¬ 
pendix  A. 

Following  experimental  administration  of  the  first  13  tests  de¬ 
veloped,  the  results  were  validated  against  officers’  ratings  of  enlisted 
radar  operators  at  8  ground  radar  centers  under  the  Tactical  Center, 
Orlando,  Fla.  Since  all  studies  of  the  validity  of  this  group  of  tests 
were  carried  out  on  ground  radar  personnel,  validity  statistics  are  not 
included  in  this  reviow.8 

Following  is  a  matrix  of  intcrcorrelations  of  12  of  the  first  13  tests 
•described  above;  tho  exception  is  the  Course  Location  Test.  Tho 
Army  General  Classification  Test,  referred  to  ns  the  AGCT,  is  included 
in  this  matrix. 
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Table  10.2  presents  t lie  rotated  factor  loading  of  these  tests,  based 
,,h  the  matrix  of  intoreorrolalions  ill  table  10.1. 
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1.  Factor  I,  best  defined  by  Form  Detection,  Oscilloscopes  Heading, 
and  Form  Conversion  I  and  IT.  appears  to  be  the  ability  to  note  quickly 
mid  discriminate  details  in  vise  tl  patterns.  This  is  the  factor  usually 
called  perceptual  speed,  found  in  nil  air-crew  classification  battery 
analyses. 

2.  Factor  TI  seems  to  be  the  ability  to  recognize  previously  seen  pat¬ 
terns.  While  this  factor,  frequently  called  visual  memory,  has  never 
been  definitely  isolated  and  pure  measures  are  not  known,  it  is  believed 
to  be  identical  with  Memory  II  identified  in  nir-crcw  classification 
battery  analyses.  None  of  the  present  tests  have  loadings  over  0.50  in 
this  factor.  Form  Conversion  I,  Coordinate  Heading,  Polar  Grid  Co¬ 
ordinate,  and  the  Army  General  Classification  Tost  have  the  highest 
loadings. 

0.  Factor  III  seems  to  be  the  ability  to  movo  one’s  self  mentally  in 
spa ce  and  predict  the  result  of . such  movement,  in  terms  of  position,  view 
of  terrain,  etc.  While  this  factor  is  here  labelled  spatial  relations  it  is 
probably  identified  with  Space  I  found  in  all  air  crew  classification 
battery  analyses.  All  the  present  tests  have  relatively  small  loadings 
with  Coordinate  Plotting,  'Parget  Course  Analysis,  and  Polar  Grid 
Coordinate  having  the  highest  loadings. 

■h  Factor  IV  best  represented  by  the  Army  General  Classification 
Tc,t  and  Scale  Heading,  seems  to  he  the  ability  involved  in  carrying 
out  ;  implo  arithmetic  computations.  This  is  the  well  known  numerical 
facility  factor. 

5.  Factor  V  appears  to  be  tlio  ability  to  estimate  lengths  without 
(lie  aid  of  measuring  devices.  The  highest  loadings  bidoug  to  Co¬ 
ordinate  Heading,  Plot  Heading,  Coordinate  Plotting,  Spot  Location, 
and  Polar  Grid  Coordinate.  This  factor  is  probably  identical  with  a 
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tentatively  defined  length  estimation  factor  isolated  in  analyses  of  tho 
air  crew  classification  batteries. 

Later  projects  concerned  with  radar  observer  selection  testing  in¬ 
corporated  several  of  the  tests  developed  by  the  NDRC  project  in 
their  selection  batteries.  The  Air-crew  Evaluation  and  Research 
Detachment  No.  1  included  two  such  te-ts  O.-eilloscope  Interpretation 
and  Cooidinate  Heading,  in  the  experimental  battery  they  admin¬ 
istered  to  Eighth  Air  Force  radar  observer  students.  Psychological 
Research  Project  (Navigator)  formed  a  selection  battery  consisting 
of  tin;  Oscilloscope  Interpretation,  Seale  Reading,  and  Polar  Grid  Co¬ 
ordinate  Tests,  which  hey  administered  at  advanced  navigator 
schools.  The  same  (  tests  which  the  AERD  No.  1  had  validated 
in  England  were  lat.  r  utilized  by  the  teams  administering  the  radar 
observer  selection  battery  to  bombardiers  and  navigators  in  the  Train¬ 
ing  Command.  Psychological  Research  Project  (Radar)  included 
the  Scale  Reading,  Polar  Grid  Coordinate,  Ratio  Estimation,  and 
Spot  Location  Tests  in  a  battery  administered  experimentally  to  radar 
observer  students. 


SELECTION  RESEARCH  IJY  THE  AIRCREW  EVALUATION 
AN1)  RESEARCH  DETACHMENT  NO.  I9 


The  AERD  No.  1,  consisting  of  G  officers  and  15  enlisted  men,  spent 
mouths  on  temporary  duty  with  the  Eighth  Air  Force  in  England 
during  the  summer  of  1914.  The  purpose  of  this  detachment  was 
twofold:  first,  to  conduct  research  activities  in  connection  with  tho 
selection  of  lead  personnel  for  very  heavy  bombardment  aircraft, 
and  second,  to  validate  air-crew  classification  test  data  in  n  combat 
theater.  The  subscipient  radar  .selection  research  which  the  AERD 
No.  1  conducted  was  in  partial  pursuance  of  this  first  objective. 

At  the  time  AERD  No.  1  arrived  in  England,  the  demand  for  radar 
observers  was  steadily  increasing.  The  number  of  radar  observers 
nipiired  by  the  operational  groups  was  of  such  magnitude  that  few 
stud  nts  could  be  eliminated  onco  they  wero  entered  in  tho  Eighth 
Air  Force  radar  observer  school.  Students  enrolled  in  tho  school 
came  from  either  tho  operational  groups  of  the  Eighth  Air  Force  or 
the  radar  training  schools  at  Langley  Field  and  Boca  Raton.  Tho 
former  group  bad  little,  if  any,  radar  training  before  entering  tho 
school.  Interviews  made  by  AERD  No.  1  revealed  varying  criteria 
for  selecting  such  officers  for  radar  observer  training.  Some,  it  was 
found,  were  sent  to  radar  school  because  they  were  regarded  as  being 
generally  competent.  Others  were  sent  because  they  wero  not  mem- 
hoi’s  of  a  crew.  Still  others  were  selected  because  they  wero  surplus. 


•  l-’or  a  more  complete  report.  r.oe  Tcplcy,  W.  M.,  td.,  op.  cit. 
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Tin'  .'•rli'H ion  of  men  arriving  from  Langley  Field  or  Boca  Baton 
was  equally  subjective  and  varied. 

In  view  of  tliis  situation,  inquiry  was  made  into  the  possibility  of 
making  preliminary  recommendations  on  selection  before  u  radar 
Ranine  became  available.  A  fact  often  stressed  by  experienced  radar 
observers  was  that  the  ellicient  use  of  radar  techniques  in  combat 
operations  depended  upon  skill  in  dead-reckoning  navigation.  They 
emphasized  that  the  radar  observer's  place  as  a  member  of  a  lead 
crew  warranted  selection  of  men  with  superior  aptitude  for  naviga¬ 
tion  as  well  as  ability  to  operate  radar  equipment. 

As  a  result,  AERO  No.  1  decided  to  investigate  first  the  navigation 
aptitude  of  current  radar  observer  students.  Navigation  stanines  were 
obtained  from  cadet  classification  records  for  students  in  the  radar 
observer  class  of  June  1011.  The  average  of  these  stanines  was  found 
to  he  0.4.  The  average  navigator  stanine  of  Navigation  school  gradu¬ 
ates  in  the  AAF  Training  Command  in  the  winter  of  19-1*2-13  was 
approximately  7.1  and  during  the  winter  of  1013— 1 1,  approximately 
8.0.  From  these  figures  it  was  apparent  that  men  selected  for  radar 
training  were  somewhat  lower  in  aptitude  for  navigation  than  their 
contemporary  graduates  from  navigation  training.  This  finding  was 
discussed  with  the  training  directors  of  the  Eighth  Air  Force  and  tho 
radar  observer  school.  On  the  basis  of  these  discussions  two  steps 
were  taken  to  insure  a  better  selection  of  potential  radar  observers 
in  the  period  before  a  radar  stanine  could  he  developed.  First,  a 
TWX  was  coordinated  with  United  States  Strategic  Air  Forces  and 
sent  by  Headquarters,  Eighth  Air  Force,  advising  Headquarters,  AAF, 
that  to  insure  successful  combat  operation  of  radar  equipment  a  mini¬ 
mum  navigator  stanine  of  eight  was  considered  essential.  Second, 
arrangements  were  made  to  have  1 1  navigators  and  bombardiers  with 
navigator  stanines  of  8  and  9  sent  to  the  radar  oh  'Tver  school  from 
one  of  the  Eighth  Air  Force  Replacement  Centers.  Eighteen  officers 
from  this  source,  nine  navigators  and  nine  bombardiers,  arrived  for 
instruction  in  the  .Inly  class. 

Following  these  preliminary  recommendations,  tho  AERD  No.  1 
turned  to  the  preparation  of  un  experimental  battery  of  tests  for  tho 
selection  of  radar  observers.  Personnel  of  the  unit  had  already  be¬ 
come  familiar. with  the  radar  observer's  job  and  equipment.  Ily  going 
on  training  missions,  observing  ground  trainers  which  simulated  air- 
borne  radar  equipment,  conferring  with  instructors,  and  studying 
manuals  and  other  training  literature,  sufficient  information  was 
obtained  to  permit  description  of  tho  work  in  general  terms  and  to 
provide  n  basis  for  choosing  tests  for  an  experimental  battery. 

The  job  description  suggested  that  the  skills  of  tho  radar  observer 
could  be  divided  roughly  into  three  categories:  skill  in  operating  and 
tuning,  skill  in  interpreting  the  oscilloscope,  and  skill  in  dead  rcckon- 
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ing  navigation.  Those  throe  categories  of  >!c ills  were  fin  (her  refined 
to  indicate  the  more  specific  aptitude.-,  and  abilities  needed  for  the 
job.  Skill  in  operating  and  tuning  the  apparatus  was  believed  de¬ 
pendent  upon  the  operator’s  familiarity  with  mechanical  devices:,  his 
speed  of  perception,  and  his  motor  coordination.  Mechanical  Prin¬ 
ciples,  CIG08B,  was  selected  to  test  familiarity  with  mechanical  de¬ 
vices;  Spatial  Orientation  I,  CPoOlli,  was  Jiosen  as  test  of  speed 
of  perception:  classification  test  scores  on  six  p>ycliomotor  tests  were 
obtained  for  as  many  students  as  possible  to  give  a  measure  of  motor 
coordination.  Skill  in  interpreting  the  oscilloscope  screen,  the  second 
category,  seemed  to  demand  proficiency  in  matching  elements  from 
a  map  with  those  on  n  screen,  in  perceiving  patterns,  in  determining 
distance  and  direction,  and  in  interpreting  the  oscilloscope  when 
jammed  with  interfering  patterns.  The  operation  of  matching  ele¬ 
ments  from  a  map  with  those  on  a  screen  suggested  the  use  of  Spatial 
Orientation  II,  CP503B.  Three  new  tests  developed  by  the  AERD 
No,  1  were  used  to  test  speed  of  pattern  perception :  Pattern  Identifica¬ 
tion  (X-I,  AERD),  Pattern  Orientation  (X-l,  KTO-AERD),  and 
Orientation  to  Landmarks  (KTO-AERD).  Coordinate  Reading 
(NDRC)  and  Dial  and  Table  Reading,  CPG22A,  wero  selected  to 
measure  ability  at  determining  direction  and  distance.  Interpret¬ 
ing  tho  oscilloscope  through  interfering  material  suggested  the  use 
of  Gottschaldt  figures,  AC  12  J,  and  Oscilloscope  Interpretntion 
(NDRC).  Skill  in  dead  reckoning  navigation,  tho  last  of  the  three 
categories  of  skill,  emphasized  speed  and  accuracy  of  computation  in- 
vol.  ing  tho  E-GB  computer  and  uirplots.  To  measure  tho  abilities 
known  to  bo  important  for  success  in  navigation  training,  tho  follow¬ 
ing  tests  wero  selected:  Numerical  Operations.  CI702A,  B;  Mathe¬ 
matics  B,  CI20GC;  Reading  Comprehension,  Cl  Bill;  and  Dial  and 
Table  Reading,  CPG22A.  Also,  nn  attempt  was  made  to  measure  di¬ 
rectly  skill  at  dead  reckoning  by  administering  n  test  of  navigation 
proficiency  developed  by  AERD  No.  1. 

In  the  foregoing  discussion,  mention  was  made  of  four  new  tests 
developed  by  AERD  No.  1:  Pattern  Identification,  CP820A,  Pattern 
Orientation,  CPS1GA,  Navigation  Proficiency,  and  Orientation  to 
Landmarks  (X-l,  ETO-AERD).  A  description  of  tho  first  two  of 
these  tests  may  be  found  in  appendix  A  of  this  report:  the  Navigation 
Proficiency  Test  is  described  in  chapter  5;  Orientation  to  Landmarks 
is  described  below. 

Tho  Orientation  to  Landmarks  Test  was  designed  to  measure  tho 
ability  to  reorientate  oneself  quickly  to  a  spatial  pattern,  seen  first  from 
nuo  direction  and  then  from  another.  Construction  of  tho  test  was 
begun  nt  Psychological  Research  Unit  No.  3.  It  consists  of  pairs  of 
aerial  photographs,  one  referred  to  as  the  reconnaissance  photograph 
and  tho  other  as  tho  cockpit  view.  In  tho  reconnaissance  photograph 
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die  aerial  view  is  always  a  vertical  one.  T n  llio  cockpit  view  the  urea 
i  photographed  from  an  oblique  angle.  Five  landmarks  are  encircled 
iiinl  ninnhered  in  each  reconnaissance  photograph.  In  the  cockpit 
view  the  same  5  points,  along  with  10  additional  points,  are  labeled 
with  the  letters  A  through  O.  The  answer  for  each  item  is  the  letter 
labeling  the  landmark  which  matches  it.  The  test  is  divided  into 
two  separately  timed  parts,  each  containing  ‘Jf>  items. 

Early  work  with  the  Orientation  to  Landmarks  Tost,  suggested  that 
the  initial  orientation  to  the  oblique  photograph  was  accomplished 
by  checking  the  relationship  of  outstanding  landmarks  to  one  another. 
It  seemed  probable  that  this  was  a  type  of  pattern  perception  which 
was  essentially  similar  to  that  involved  in  matching  patterns  from 
map  to  oscilloscope  screen. 

Following  the  selection  of  tests  for  the  experimental  battery,  AERD 
No.  1  was  faced  with  the  problem  of  selecting  a  criterion  for  validation 
purposes.  Several  criteria  were  considered  and  rejected,  somo  be¬ 
cause  complete  records  were  not  available  and  some  for  other  reasons. 
For  example,  records  of  practice  bombing  missions  mid  camera  mis¬ 
sions  were  incomplete  because  cloud  formations  frequently  interfered 
with  tho  taking  of  photogi  aphs.  Results  of  combat  missions  wero 
rejected  for  the  same  reason.  An  attempt  was  made  to  obtain  ratings 
on  radar  observers  in  five  groups  of  the  Third  Division  of  the  Eighth 
Air  Force.  The  effort  was  abandoned  ns  impractical  because  only  a 
few  had  been  rated,  most  of  the  observers  having  boon  just  recently 
assigned  to  tho  group.  Pass- fail  criterion  in  tho  radar  observer 
school  was  discarded  because,  ns  mentioned  before,  the  needs  of  tho 
operational  groups  were  such  ns  io  prevent  nay  substantial  number 
of  students  being  failed.  After  eliminating  these  possibilities,  AERD 
No.  1  decided  to  use  us  a  criterion  composite  grades  in  radar  observer 
sclmo!.  The  composite  grade  was  a  mean  of  grades  obtained  by 
assigning  a  weight  of  2  to  the  mean  flight-  grade  and  a  weight  of  1 
to  the  mean  ground  trainer  grade.  Usually  the  mean  flight  grndo 
cnnsFted  of  4  flight  grades;  the  mean  ground  grade  was  composed 
of  approximately  8  to  12  ground  grades. 

Table  10.8  presents  correlations  between  the  composite  grade,  and 
the  tests  administered  to  classes  -IT — 1  and  Tl-.r>.  These  correlations 
were  Feed  only  upon  students  from  Langley  Field  and  Pocu  Raton 
because  it  was  expected  that  subsequent  classes  nt  the  school  would  bo 
eninpo.-ed  entirely  of  such  students.  In  the  statistical  treatment  of 
the  data  which  follows,  these  students  will  he  referred  to  as  the  Lang¬ 
ley  group.  Table  10.8  indicates  that  Mio  following  seven  tc.  ts  had 
Correlations  of  suflicient  magnitude  with  the  compo.-ite  grade  to  bo 
of  predictive  value:  O.  cilloseopo  Interpretation,  Coordinate  Reading 
I  and  II,  Spatial  Orientation  I,  Pattern  Orientation,  Two-IIand 
Coordination,  and  Complex  Coordination.  Since  the  intercorrela- 
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t ion  of  Pin  ts  T  and  II  of  Coordinate  Heading  'vus  0.74  they  wero 
combined  in  :i  single  score. 


Tami.k  10. .'{ — Minus,  stmntnnl  <h  rinlions,  unit  corrrln I  ions  of  tests  with  o>»i- 
posilc  gnnlrs  fur  Langtry  turn  only  of  vtassrs  -}-}  --}  unit 
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It  will  bo  noted  tlmt  tho  covrelntion  of  composite  grade  with  navi¬ 
gator  stnnine  is  0.05,  with  Navigation  Proficiency  Test,  Part  I  (pilot- 
ngo  and  instruments)  0.15,  and  with  Navigation  Proficiency  Test, 
Part  II  (E-OB  computer),  0.07.  This  seemed  to  provide  a  clear  dem¬ 
onstration  that  composite  grade  for  the  type  of  training  given  at  tho 
radar  observer  school  was  u  function  of  variables  other  than  naviga¬ 
tional  aptitude  and  proficiency.  Persons  conducting  tho  study  em¬ 
phasize  that  this  should  not  be  misinterpreted  as  evidence  against  tho 
po.  it  ion  stated  earlier  that  navigators  of  high  ability  should  bo  se¬ 
lected  for  radar  observer  training  since  the  school  criterion  did  not 
require,  the  degree  of  navigational  skill  called  for  under  operational 
conditions. 

The  (3  tests  most  highly  correlated  with  composite  grades  were  in- 
tercorrelated.  The  intereoi relations  based  on  classes  •If— 1  and  44-5 
are  presented  in  table  10.  t.  Tho  N  varies  from  85  to  170  beeauso  not 
all  of  the  tests  wore  administered  to  all  classes. 
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A  Her  talcing  account  o  f  magi  lit  n«l«*  of  i ill  m  correla  t  ions,  rm re hit  ions 
•vs  i 1 1 1  tlir  composite  grade,  and  standard  deviations,  tin-  tots  wero 
a.-si^t n’d  tentative  weights  as  follows; 


Test :  "’'W 

Osillleseojii*  Interpretation _ 1 

On>rdiua,tc  H'MiliiiK  _ 1 

Spatial  Oriental Inn  I _  1 

Pattern  Orientation _  2 


On  the  basis  of  this  weight ing,  tin  aggregate  score  was  obtained  for 
each  student  in  class  44-G,  and  u  prediction  made  of  composite,  grado 
in  terms  of  above  average,  average,  and  below  average.  The  two 
psyehomotor  tests  were  not  included.  Scores  for  them  were  available) 
for  only  approximately  50  percent  of  class  1 1-G.  The  results  wero  ns 
follows: 

Above-average  group:  Of  this  group  74  percent  were  nbovo  tho 
median  composite  grade,  while  2G  percent  were  below  this  grade. 

Two  students  were  eliminated. 

Average  group:  Twenty-c’  :ht  percent  of  this  group  were  abovo 
the  median  composite  grade,  72  percent  were  below,  mul  one  stu¬ 
dent  was  eliminated. 

Below-nverage  group:  Thirty  percent  of  this  group  wero  nbovo 
the  median  composite  grado  and  70  percent  were  below.  Eight 
students  were  eliminated. 

Thus,  if  the  lower  third  on  the  tests  had  been  excluded  from  training, 
only  3  instead  of  11  students  would  have  been  eliminated. 

Students  from  class  44— G  were  added  to  the  previous  classes  and 
validities  on  the  six  most  promising  tests  calculated  for  tho  total 
numbers  of  available  cases.  The  results  are  shown  in  table  10.5. 


T.Uil.K  10.,r). — Validity  cneflleicnt*  and  S'.i  (or  the  .its  text*  of  hiyhr.it  validity  in 
Air-Crew  I-Jraluation  AV.muv/t  Detachment  .Vo.  1  ttndy 
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Test  :  (peremt) 

Coordinate  Heading  (NDItC) -  "  < 

Two  Hand  Coordination.  CI1MA _  1!:) 

Complex  Coonlih.'tllon,  CM701A -  21 

^p  illal  Orientation  I,  Cl’oOIlt -  14 


Using  these  data,  Psychological  Research  Project  (Radar)  later 
found  that  the  correlation  of  the  be.-t  possible  weighted  combination 
of  these  six  tests  with  success  in  radar  observer  training  would  bo 
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0.1!).  The  In*. si  possible  weighted  combinat ’.  >n  of  the  following  four  of 
those  tests  would  correlate  0.18  with  the  same  criterion. 

Tn  conjunction  with  its  use  of  the  composite  grade  as  a  criterion, 
AEKD  No.  1  made  several  comparisons  of  the  relative  success  of  vari¬ 
ous  groups  of  students  within  the  school.  As  de.-cribed  earlier  in  this 
chapter,  the  radar  students  tested  in  the  course  of  (his  study  fell  into 
two  primary  groups — those  with  previous  radar  training  in  the  United 
Slates  and  those  without  such  training.  The  latter,  recruited  from 
operational  groups,  will  he  referred  to  in  discussing  the.  statistical 
treatment  of  the  data  as  the  non-Langley  sample.  Students  receiving 
radar  ob.-erver  training  also  diUVrcd  with  respect  to  their  classification 
as  either  navigators  or  bombardiers.  Data  concerning  such  classifica¬ 
tion  were  avnilablo  for  classes  •11— 1,  11-5,  and  11—6. 

For  the  four  classes  shown  in  table  10. G  it  will  be  seen  that  there  are 
only  small  differences  in  composite  grades  between  students  with  pre¬ 
vious  radar  training  at  Langley  and  those  without  The  greatest 
difference  is  in  class  11-4.  Here  the  biscrinl  correlation  of  composite 
grado  with  prcsouco  or  absence  of  Langley  training  was  0.3G.  For 
class  11-5  the  bi.scrial  correlation  of  the  samo  variables  was  0.0G. 
Of  intciost  in  class  11-5  was  the  relatively  favorable  showing  of 
the  18  non-Langley  students.  As  pointed  out  earlier  those  students 
were  selected  to  fill  out  class  11-5  on  the  assumption  that  only  men 
having  the  highest  aptitude  for  general  navigation  should  receive 
training  in  the  ANAA.PS-15  operation.  Their  mean  composite  grado 
was  as  high  as  that  of  students  with  previous  radar  experience  or 
combat  navigation. 


Taui.k  10.0. — .1  feans  and  standard  deviations  of  composite  grades,  Langley  and 
non-Langley  students,  for  each  class  in  Eighth  Air  Force  Radar  Observer 
School 
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•  6|Wc!nUy  svlvrlod  lil^li  slnnlno  navigators  and  bombardiers. 


Table  10.7  presents  the  mean  composite  grades  and  standard  devia¬ 
tions  for  the  navigators  and  bombardiers  of  the  Langley  group  for  two 
classes.  Examination  of  this  table  indicates  that  tbo  two  categories  of 
students  did  equally  well.  This  comparison  is  partly  vitiated  by  the 
fact  that  n  si/.able  proportion  of  the  bombardier  students  had  received 
intensive  training  in  dead  reckoning  navigation  while  serving  as  navi¬ 
gation  instructors  in  bombardier  schools. 


'j'.v iti-K  10.7.  —  Minus  and  standard  deviations  of  composite  grades  for  navigators 
and  bombuvdii  r.i  ( I.nngh  y  students  only)  for  classes  .))-}  and  .JJ-J  of  I'.ighlh 
Air  Force  Radar  Observer  School 
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Nevertheless,  significant  differences  did  exist  between  navigators 
and  bombardiers  witii  respect  to  navigation  proficiency  ns  measured  by 
the  Navigator  Proficiency  Test  (ETO-AERD).  Table  10.8  suin- 


Taih.k  10.8. — Point  biscrial  correlations  for  classes  and  J  }-G  bchcccn  navi¬ 
gator-bombardier  classification  and  various  parts  of  the  navigator  proficiency 
test  ( FT  O- A  Kill)) 
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mariz.es  point  biscrial  correlations  between  the  navigator-bombardier 
classification  and  scores  obtained  on  the  various  parts  of  this  test. 

STUDENT  SELECTION  I5Y  PSYCHOLOGICAL  RESEARCH 
PROJECT  (NAVIGATOR) 

In  accordance  with  a  directive  from  Headquarters,  AAF,  in  July 
1911,  the  Psychological  Research  Project  (Navigator)  adapted  special 
screening  tests  developed  by  the  NDRC  project  to  aid  in  the  selection 
of  individuals  for  radar  training  in  the  AAF  Training  Command. 
At  this  time,  students  being  trained  were  almost  exclusively  bom¬ 
bardiers,  but  upon  the  recommendation  of  Headquarters,  Eighth  Air 
Force  that  only  men  with  navigator  stall ines  of  eight  or  inoro  bo 
given  radar  oh-erver  training,  it  was  decided  that  these  tests  bo 
administered  in  the  advanced  navigator  schools. 

Tlio  selection  battery  decided  upon  consisted  of  tlireo  tests  de¬ 
veloped  by  the  NDRC  project.  They  were  Oscilloscope  Interpretation, 
CPtfl7A,  Seale  Reading,  CPfi-’/fA,  and  tlio  Polar  Grid  Coordinate 
Test.,  CPS19B.  These  tests  were  adapted  to  machine  scoring  which 
involved  minor  changes  in  a  few  of  the  items.  Additional  criteria  for 
selection  were  a  strong  preference  for  this  typo  of  training  and  a. 
navigator  stannic  of  eight  or  nine.  Preference  for  radar  observer 
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trainin''  was  indicated  by  choosing  on**  of  four  statements  arranged 
in  diminishing  interest  order  from  one  to  four  (i.  c.,  “1”  represented 
highest  degree  of  interest,  “  P  the  lowest) . 

In  August  and  .September  of  191  f,  representatives  of  the  Navigator 
Project  administered  tlie  test  battery  at-  Sail  Marcos,  Hondo,  and 
Ellington  Air  Fields.  A  fter  the  initial  testing  at  each  of  the  advanced 
schools,  suitable  representatives  of  each  school  were  designated  to 
carry  on  the  testing  in  order  to  allow  the  project  to  concentrate  on 
its  research  activities.  However,  the  Navigator  Project  still  main¬ 
tained  a  general,  supervisory  responsibility  with  respect  to  this  work. 

At  each  school  a  roster  of  students  meeting  the  -  tanine  and  prefer¬ 
ence  requirements  was  prepared  listing  them  in  order  of  their  radar 
aptitude  test  scores.  The  radar  aptitude  score  was  a  composite  score 
arrived  at  by  adding  the  raw  scores  on  the  Oscilloscope  Interpretation 
(CP817A)  and  Scale  Reading  (CPG37A)  tests  to  one-half  the  raw 
score  on  the  Polar  Grid  Coordinate  Test  (CP819B).  Radar  observer 
schools  quotas  were  filled  hv  selecting  men  from  this  roster  in  order 
of  their  radar  aptitude  score. 

The  Navigator  Project  terminated  radar  observer  selection  activi¬ 
ties  when  Headquarters,  AAF  activated  test  teams  for  the  specific 
purpose  of  radar  selection  testing. 

STUDENT  SELECTION  BY  HEADQUARTERS,  AAF  TRAINING 

COMMAND 

A  directive  from  Headquarters,  AAF  Training  Command,  Fort 
Worth,  Tex.,  in  September  1914,  established  six  traveling  testing 
teams,  two  in  each  Training  Command,  to  carry  out  the  selection 
testing  begun  by  the  Navigator  Project.  The  number  of  teams  was 
reduced  to  two  in  January  1945.  Each  team  was  composed  of  an 
orientation  officer  who  was  a  combat  returnee,  an  aviation  psychologist, 
and  two  psychological  assistants.  When  selection  of  potential  radar 
observers  was  first  undertaken  by  these  teams,  only  rated  pilots  were 
tested.  After  11  December  1944,  pilots  were  replaced  by  rated 
bombardiers  and  navigators. 

The  test  battery  administered  by  the  testing  teams  consisted  of 
four  of  the  same  tests  which  AERD  No.  1  had  administered  to  stu¬ 
dents  in  the  radar  observer  school  of  the  Eighth  Air  Force.  They 
wore:  Pattern  Orientation,  CPS1GA;  Coordinate  Reading,  CP224A,  B; 
Oscilloscope  Interpretation,  CP817A;  and  Spatial  Orientation  I, 
CP501B.  The  battery  also  included  a  nine  point  numerical  interest 
scale  ranging  from  “1”  which  represented  little  or  no  interest,  to  “9” 
which  indicated  exceptionally  strong  interest. 

A  composite  radar  score  was  computed  by  adding  the  raw  scores 
on  tho  four  tests.  Since  the  standard  deviation  of  the  Coordinate 
Reading  Test  was  approximately  twice  the  size  of  each  of  the  other 
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three.  tests,  this  test  received  a  correspondingly  high  weight, 
practical  use  the  composite  score  was  converted  to  a  stanine. 

Navigators  and  bombardiers  assigned  to  radar  training  were  selected 
in  order  of  their  radar  observer  aptitude  stanines.  To  be  eligible, 
a  radar  observer  stanine  of  six  or  above  was  required.  In  addition, 
a  navigator  stanine  of  at  least  seven  and  an  interest  preference  of 
three  or  above  were  required.  Provisions  were  made,  however,  to 
progressively  lower  these  standards  if  higher  quotas  necessitated  the 
change.  After  1  May  1015,  most  of  the  assignments  of  bombardiers 
to  radar  observer  training  were  made  from  the  upper  half  of  the 
distribution  on  a  standardized  navigation  proficiency  test.  This 
selection  requirement  operated  in  addition  to  those  previously 
mentioned. 

The  table  below  summarizes  the  recommendations  of  bombardiers 
and  navigators  to  radar  observer  school  for  the  period  9  November 
1911  through  30  Juno  1945.  Because,  of  an  insufficient  number  of 
bombardiers  and  navigators  available  for  radar  observer  training, 
selection  testing  was  terminated  31  July  1945. 

Taw.k  10.9. — Nummary  o/  radar  observer  screening  and  recommendations  {or 
the  period  9  November  19H  through  SO  June  10i5 


Total  number  of  bombardiers  screened _ _ _  0,057 

Total  number  of  navigators  screened _  0,859 


Grand  total  screened _  13,810 


Total  number  of  bombardiers  recommended _  2,051 

Total  number  of  navigators  recommended _  1,988 


Grand  total  recommended _  -1,039 

lac  bombardiers  recommended  Included: 

J’crcml 

Combat  returnee  volunteers -  4.  7 

Permanent  party - 9.1 

Recent  graduates -  80.2 

The  navigators  recommended  Included: 

Combat  returnee  volunteers -  3.3 

Permanent  party _  4.4 

Recent  graduates _  02.3 


Preparations  were  made  in  May  1945,  to  extend  tho  selection  of 
nular-observer  students  to  unclassified  cadets.  At  tho  request  of 
Headquarters,  AAF,  Psychological  Research  Project  (Radar)  formu¬ 
lated  recommendations  for  tests  and  weights  upon  which  a  radar  ob¬ 
server  stanine  was  to  be  based  in  classification  centers. 

Previously,  tho  Radar  Project  had  found  the  best  possiblo  weighted 
combination  of  4  of  the  0  tests  with  appreciable  validity  in  the  A  HUD 
No.  1  study  to  bo  ns  follows: 
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Wright, 

Test : 

percentage 

<  ’oonlln.'tte  Iti-adlng,  <''1>22  IA,  B  _  _  _  _  3(5 

Twd-1  land  Coordination,  (TI01A _  _  _  _  20 

Complex  Coorillimllon,  C.M701A _  _ _  21. 

Spatial  Orientation  I,  CI*5M1I1 _ _ _  14 


While  baling  it.-,  recommendations  primarily  upon  the  validity  sta¬ 
tistics  reported  by  AERD  No.  1  the  project  took  note  of  the  fact  that 
certain  tests  then  in  the  classification  battery  had  not  been  available 
to  AKRD  No.  1  at  the  time  of  its  validation  studies.  It  was  felt,  more¬ 
over,  that  class  grades  used  as  a  criterion  in  the  AKRD  No.  1  study 
were  determined  primarily  by  the  ability  of  the  radar  observer  to  use 
the  radar  set  and  did  not  reflect  the  navigation  and  bombing  require¬ 
ments  which  would  he  present  in  the  Training  Command  criterion. 
A  summary  of  the  test  recommended  and  the  weights  assigned  to  each 


is  presented  below: 

Weight, 

Tost  :  percentage 

Complex  Coordination,  CM701A _  20 

Two-Iln nd  Coordination,  CM101A _  20 

Instrument  Comprehension  II,  C1010B -  10 

SpaHnl  Orientation  I,  Cl’SOlB _  10 

Coordinate  Heading,  01*221  A,  B . . - .  20 

Dial  and  Table  Beading,  CTG22.V  and  021A _  10 

Ariilmietlc  Reasoning,  CI20GC _  10 


In  the  original  combination  of  four  tests  listed  above,  50  percent 
of  the  weight  had  been  assigned  to  factors  measured  by  Two-IIand 
Coordinnlion,  CM101A  and  Complex  Coordination,  CMT01A.  In¬ 
strument  Comprehension  II,  C1010B,  however,  was  not  in  the  classi¬ 
fication  battery  at  the  time  air  crew  officers  in  this  sample  were  classi¬ 
fied.  Factor  analyses  completed  at  the  Psychological  Research  Unit 
indicated  that  Instrument  Comprehension  II,  CIG1GB,  was  heavily 
loaded  with  a  factor  which  was  also  an  important  component  in  Com¬ 
plex  Coordination,  CM701A,  and  Two-IIand  Coordination,  CM101A. 
This  is  the  ability  called  spatial  relations  in  chapter  4.  Job  analysis 
led  to  the  conclusion  that  this  was  one  of  the  more  important  abilities 
determining  success  in  the  radar  observer’s  task.  As  a  result,  a  weight 
of  10  percent  was  recommended  for  Instrument  Comprehension  II, 
and  20  percent  each  for  Two-IIand  Coordination  and  Complex  Co¬ 
ordination. 

It  wns  recommended  that  Arithmetic  Reasoning,  CI20GC,  and  Dial 
and  Table  Reading,  CPG22A  and  G21A,  both  tests  of  navigational 
aptitude,  bo  included  in  tho  stnnino  because  of  tho  conviction  that, 
against  a  Training  Command  criterion,  abilities  important  to  naviga¬ 
tion  would  also  determine  success  for  radar  observers.  For  each  of 
these  tests  a  weight  of  10  percent  was  suggested.  Somo  of  the  weight 
indicated  by  the  AERD  No.  1  results  for  Coordinate  Reading,  CP224A, 
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B,  was  assigned  to  those  tests  partly  because  of  the  correlation  of  0.70 
f, , mid  between  Coordinate  Heading  and  Dial  and  Table  Heading  in 
the  AERD  No.  1  study.  The  weight  assigned  to  Spatial  Orientation 
I,  CT.T1B,  was  reduced  from  I  t  percent  to  10  percent  because  of  the 
fact  that  two  of  the  tests  added.  Instrument  Comprehension  II  and 
Dial  and  'Table  Reading  had  substantial  loadings  in  perceptual  speed. 
Finally,  it  was  recommended  that  Coordinate  Reading,  CRAM  A,  H, 
he  added  to  the  elassifieat ion  battery  with  a  suggested  weight  of  10 
percent  for  the  radar  stanine. 

After  1  June  10  la,  all  students  taking  the  air-crew  classification  tests 
were  given  a  radar  stanine  computed  in  terms  of  these  recommenda¬ 
tions.  However,  because  of  the  termination  of  hostilities,  no  students 
were  assigned  to  training  on  this  basis. 

SUMMARY 

The  chief  contributors  to  the  development  of  a  selection  battery  for 
radar  observers  were  Project  SC-70,  NS-llG,  of  the  National  Defenso 
Research  Committee  and  the  Air-crew  Evaluation  and  Research  De¬ 
tachment  No.  1.  The  NDRC  project  conducted  the  initial  research 
in  the  field,  and  prepared  a  number  of  tests  for  purposes  of  radar 
selection.  Three  of  these  tests,  Oscilloscope  Interpretation,  CP817A, 
Scale  Reading,  CPG37A,  and  the  Polar  Grid  Coordinate  Test,  CPS19B, 
comprised  the  radar  observer  selection  test  buttery  administered  by 
Psychological  Research  Project  (Navigator).  Oscilloscope  Interpre¬ 
tation,  CPS17A,  and  Coordinate  Reading,  CP221A,  B,  wero  used 
in  the,  experimental  battery  administered  by  AERD  No.  1  and  also  by 
traveling  teams  administering  the  radar  selection  battery  at  advanced 
navigator  and  bombardier  schools.  Coordinate  Reading  was  later 
included  in  the  Air-crew  Classification  Battery  to  aid  in  determining 
a  radar  stanine  for  students  taking  the  classification  tests. 

As  a  part  of  its  broad  assignment  to  research  on  lead  crew  selection, 
AERI)  No.  1  validated  nil  experimental  battery  of  potential  radar 
observer  aptitude  tests.  Of  the  21  tests  which  AERD  No.  1  validated 
against  composite  grades  in  radar  observer  school,  0  had  validity 
coefficients  of  sufficient  magnitude  to  be  of  predictive  value.  They 
were:  Oscilloscope  Interpretation,  CPS17A,  Coordinate  Reading, 
CP22IA,  B,  Spatial  Orientation  I,  CP501B,  Pattern  Orientation, 
CPvSIGA,  'Two-IIand  Coordination,  CP221A,  B,  and  Complex  Coor¬ 
dination.  CM701A.  The  best  possible  weighted  combination  of  tlieso 
mx  tests  had  a  correlation  of  0.-19  with  success  in  radar  observer  train¬ 
ing.  Later,  four  of  these  tests,  Oscilloscope  Interpretation,  Coordi¬ 
nate  Reading,  Spatial  Orientation  I,  and  Pattern  Orientation 
formed  the  Radar  Observer  Selection  Battery  administered  by  travel¬ 
ing  teams  at  advanced  navigator  and  bombardier  schools  in  tho  train¬ 
ing  command. 
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1  *>ycl»t)lo*;iciil  Research  Project  (Navigator)  conducted  the  first 
selection  testing  for  the  purpose  of  a.—  igning  students  to  radar  ob¬ 
server  training.  The  selection  battery  used  has  been  listed  above. 
When  Headquarters,  AAF,  expanded  the  selection  program  and 
activated  traveling  teams  to  test  students  at  advanced  navigator  and 
bombardier  schools,  the  Navigator  Project  terminated  its  radar  ob¬ 
server  selection  activities. 

The  teams  activated  by  Headquarters,  AAF,  in  September  1944, 
administered  a  selection  battery  of  four  tests  which  A  HIM)  No.  1  had 
validated  at  the  Eighth  Air  Force  Radar  Observer  School  in  England. 
Requirements  for  assignment  to  radar  school  were  a  radar  stanine  of 
six  or  more  and  a  navigator  stanine  of  seven  or  above.  The  radar 
stanine  was  derived  from  the  student’s  scores  on  the  selection  test  bat¬ 
tery  and  an  interest  indication  of  a t  least  three  on  a  nine-point  scale. 
After  1  May  194a,  a  navigation  proficiency  test  was  included  in  the 
battery  administered  at  advanced  bombardier  schools. 

Psychological  Research  Project  (Radar)  in  May  1913  made  a  num¬ 
ber  of  recommendations  relative  to  the  tests  and  weights  to  be  used 
in  determination  of  a  radar  stanine  from  air-crew  classification  test 
scores.  These  recommendations  were  based  partly  on  validity  statis¬ 
tics  reported  by  AERD  No.  1  and  partly  on  other  considerations.  The 
tests  and  their  weights  were  as  follows: 


Wclghtt 

Test:  ( percentage ) 

Complex  Coordination,  CM701A -  20 

Two-IIand  Coordination,  CI101A _  20  • 

Instrument  Comprehension  II,  CI010B _  * 0 

Spntlnl  Orientation  I,  CPG01B _ _ ___  10 

Coordinate  Heading,  CP22-IA,  B _  20 

I)l:il  mid  Table  Heading,  CP022A -  10 

Arithmetic  Hoasonlng,  CI20CC -  10 


After  1  June  1945,  alt  students  taking  the  air-crew  classification  tests 
were  given  a  radar  stanine  computed  in  terms  of  theoO  recommenda¬ 
tions.  However,  because  of  the  termination  of  host  dities,  no  students 
were  assigned  to  training  on  this  basis. 


CHAPTER  11 _ 

Validation  of  Selection  Tests  for 
Radar  Observer  Training' 


INTRODUCTION 

From  {.lie  time  of  its  activation,  the  main  goal  of  the  Psychological 
Research  Project  (Radar)  was  the  validation  of  tests  for  radar  ob¬ 
server  selection.  Even  (hough,  as  explained  in  chapter  .‘5,  initial  em¬ 
phasis  was  placed  upon  the  development  of  proficiency  measures,  it 
was  clear  that  the  primary  research  rote  of  these  measures  was  to  servo 
as  criteria  for  validating  selection  tests.  Similarly,  it  was  for  this  rea¬ 
son  primarily  that  the  analysis  of  bombing  errors  described  in  chapter 
0  was  undertaken.  This  concern  with  the  development  of  acceptable 
criteria  seemed,  in  radar  observer  training,  to  he  a  necessary  prerequi¬ 
site  to  the  potentially  more  significant  research  upon  test  validation. 

In  this  chapter  a  report  is  made  of  the  results  of  validating  various 
selection  tests  against  quantitative  job  criteria,  both  course  grades  and 
bombing  scores.  The  methods  and  results  of  two  extensive  validation 
studies  are  described.2  For  convenience,  tin*  studies  will  be  referred 
to  as  validation  studies  I  and  II.  For  each,  descriptions  are  given 
of  the  variables  validated,  the  criteria,  and  the  samples  us  well  as 
the  resulting  validity  coefficients.  In  study  II  multiple  regression 
statistics  arc  presented  for  various  combinations  of  the  variables  found 
to  have  appreciable  validity.  In  each  study  two  validation  samples 
were  used,  one  of  bombardiers  and  one  of  navigators.  The  reliability 
of  the  course  grades  was  determined  for  portions  of  the  validation 
samples.  The  estimates  of  reliability  are  presented  with  considerable 
mii-givings  but  are  included  because  they  are  the  best  estimates  avail¬ 
able  under  the  circumstances. 

The  results  of  the  two  studies  arc  discussed  jointly  at  the  end  of  the 
chapter.  An  attempt  is  made  to  summarize  the  evidence  regarding 
the  importance  of  various  abilities  to  success  in  'radar-observer  training. 

1  Written  by  Cpl.  ITarold  II.  Kelley. 

'The  following  pcr^onnpl  wrre  for  the  plftunlnjj  of  tbt%  valMflMon  j 

ltolanj  E.  Johnston,  Jr.,  Cpl.  Kellcj,  U,  Sol  M.  ltonhnl,  nntl  8/Stft.  Ucrnanl  C« 
hull  Ivan. 
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VALIDATION  STUDY  I 


Validation  study  I  consists  of  t lit'  validation  of  selection  test  scores 
which  were  available  for  radar-observer  students  before  experimen¬ 
tal  selection  testing  was  begun  by  the  Radar  Project.  The  criterion 
against,  which  the  scores  were  validated  consisted  of  the  radar-observer 
course  grades  assigned  by  the  training  stations.  'Two  samples  were 
used,  a  bombardier  sample  including  mostly  students  from  Boca  Raton 
and  Langley  Field  and  a  navigator  sample  including  mostly  students 
from  Victorville. 

Variables 

This  study  validates  the  variables  in  two  batteries  of  selection  tests: 
the  air-crew  classification  battery  (November  1913)  and  tho  radar- 
observer  selection  battery.  Appendix  A  describes  each  of  the  tests 
composing  the  two  batteries. 

The  air-crew  classification  battery  used  in  this  study  consists  of  11 
printed  tests  and  0  psychomotor  tests.  It  was  administered  to  aviation 
students  before  they  entered  prelliglit  school.  Test  scores  were  dif¬ 
ferentially  weighted  and  combined  into  three  stanines  or  standard 
scores  indicating  aptitude  for  each  of  the  three  air-crew  specialties: 
bombardier,  navigator,  and  pilot.  It  was  on  the  basis  of  stanines  that 
aviation  students  were  classified  and  assigned  to  one  of  the  three  air¬ 
crew  specialties.  Although  the  air-crew  classification  battery  was 
changed  at  different  times  as  new  tests  and  validation  information  be- 
camo  available,  variables  lor  the  present  study  are  all  taken  from  the 
November  1913  battery  which  was  administered  to  aviation  students 
from  November  1913  to  September  19-11.  Appendix  A  describes  the 
composition  of  the  three  stanines  computed  from  the  battery  in  terms 
of  how  much  each  test  contributes  to  each  stanine. 

The  radar-observer  selection  battery  consi  s  of  four  printed  tests 
and  an  interest  blank.  It  was  administered  to  bombing  students  and 
navigation  students  just  prior  to  their  being  commissioned  and  gradu¬ 
ated  from  advanced  training.  Test  scores  were  weighted  and  com¬ 
bined  into  one  stanine  for  tho  air-crew  specialty  of  radar  observer 
(bombardment).  The  composition  of  this  stanine  is  described  in  chap¬ 
ter  10.  On  page  221  is  described  the  Interest  Blank,  Radar  Preference 
i,  which  is  an  indication  of  the  strength  of  preference  for  radar-ob¬ 
server  training  expressed  at  the  time  of  taking  the  test  battery. 

In  all,  27  selection  variables  are  represented  in  validation  study  I. 
From  the  uir-crew  classification  battery  there  nre  21  variables,  includ¬ 
ing  tho  three  stanines.  From  tho  radar-observer  selection  battery  there 
nre  0  variables  including  the  ono  stanine  and  Radar  Preference 
Rating  I. 
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\  filiation  Criteria 

All  selection  variables  wore  validated  against  the  final  course  grades 
assigned  students  graduating  from  the  radar-observer  coi;;>e.  Tublo 
11.1  presents  the  specific  student  classes  from  which  the  criterion 
iriades  in  the  present  study  were  taken.  Course  grades  at  all  tliree 
Hclu-als  were  determined  as  a  composite  of  flight,  trainer,  ami  classroom 
grades.  For  all  classes  in  the  present  study  with  the  exception  of  six 
Victorvill"  classes,  the  flight  grades  contributed  GO  percent  of  the 
composite,  and  trainer  and  classroom  grades  each  contributed  “JO  per¬ 
cent.  For  Victorville  classes  15-10  through  15-15,  flight  grades  made 
up  75  percent  of  the  total  course  grades  with  the  remaining  J5  percent 
being  half  trainer  and  half  classroom  grades. 


T.viii.k  11.1. —  limlnr  observer  clause*  inter/  from  auk  uchuol  in  Yuliilutiun  Study  I 


School 

Clawcs 

Oru'hiiUlon  Oates 

<5-7  UirniiKh  15-18 . 

21  February  1915-  5  May  1915. 

21  March  l I'J  May  1915. 

10  March  1915-12  May  l‘J<5. 

i  v 3  ilirnuuli  i.5-11  _ 

IV  10  tliruuyh  <5-19 . 7. 

The  course  grades  for  these  classes  represented  ratings  given  by 
instructors,  scores  from  informal  classroom  quizzes,  and  scores  from 
standardized  tests  and  performance  checks  developed  by  the  Radar 
project.  Only  a  rough  estimate  can  bo  made  of  the  extent  to  which  tho 
three  schools  based  course  grades  upon  instructor  ratings  and  class¬ 
room  quizzes  as  compared  with  the  relatively  more  standardized  tests 
and  cheeks.  At  Boca  Raton  and  Langley  Field,  about  half  the  classes 
in  the  present  study  were  given  the  complete  battery  of  proficiency 
measures  developed  by  the  Radar  Project.  This  battery  is  described 
in  chapters  5  and  (i.  At  Victorville,  the  last  class  was  given  most  of  the 
Radar  Project  tests  and  checks  and  the  preceding  three  classes  wero 
given  almost  all  of  the  tests.  Of  the  remaining  classes  only  two 
received  one  or  more  standardized  tests. 

'flic  three  schools  made  varying  use  of  the  scores  from  such  stand- 
ardizod  measures  as  were  employed  during  this  period.  It  is  estimated 
that  for  the  Boca  Raton  and  Langley  Field  classes  in  study  I,  the  course 
grades  were  based  upon  scores  from  standardized  measures  and  infor¬ 
mal  grades  in  equal  proportion.  At  Victorville,  however,  course 
grades  were  based  almost  completely  upon  instructors’  daily  grades 
and  informal  classroom  tests. 

For  each  training  station,  a  distribution  of  course  grades  was  made 
for  “ill  students  in  the  classes  studied.3  Two  distributions  were  neces¬ 
sary  for  Victorville  due  to  a  shift  in  the  grading  system  from  class 
•15-15  to  class  -15-10.  To  make  the  scores  from  the  three  schools  coin- 

*  Tills  MntlsUcnl  work  wan  carried  out  by  Sj;t.  GeraM  S.  Ilium  nml  S/Hyt.  Harold  P. 
Kum-nmn. 
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parable,  I  In*  semes  from  Lan:_dty  Field  :  1 1 1 >  1  Victorville  were  run  verted 
to  distributions  con  e.- pm  id  mg  to  the  Boca  Baton  distribution  which 
lias  :i  mean  of  .S  1.0S  ami  a  standard  deviation  of  3.tHi.  l)i-t rihut  i mis  of 
the  converted  course  grades  w*re  made  separately  for  the  bombardier 
sample  and  navigator  sample  finally  used  in  obtaining  the  validity 
e< /cllieient s.  Statistics  from  these  distributions  are  presented  in 
table  1  l.'i.  It  may  he  noted,  incidentally,  that  on  the  average  naviga¬ 
tors  obtained  higher  course  praties  than  bombardiers,  the  difference 
bavin;.'- a  crit  ical  rat  io  of -1.33.  This  is  significant  at  the  1  percent  level. 

Taiii.k  11.2.—  Slutislics  from  ilisl ribu I ions  of  converted  course  •jrudes  for  bom- 
hnrdirr  m\d  mi  vi;/ntor  samples,  Ynlidnllon  Study  / 


Sample 

M 

81) 

lion)  .r.llrri  .  . 

VI.  17 

4.  1.1 

N.ivls  ilori .  . 

bt.Ui 

3.&5 

Critlnil  r  U  Id  of  <1  Or>-r<-n<-e  hr  wr«-n  in,-aii3  —  t.33.1 
1  ;*i|:nini*:iiit  ul  the  1-pcrrrnl  level. 


No  precise  measure  could  be  made  of  the  reliability  of  the  course 
grades  for  the  validation  samples.  However,  a  rough  estimate  was 
made,  using  the  course  grade  and  its  components  for  Lang1'  ■/  Field 
cl.is.-es  1.1-11  through  h1-df>.  Course  grades  for  these  classes  were 
based  partly  upon  standardized  tests  and  performance  checks,  and 
partly  upon  daily  grades  and  informal  tests  as  were  the  course  grades 
for  oilier  Langley  Field  and  Boca  Baton  classes  in  study  I.  Whether 
or  not  the  coefficient  obtained  is  a  fair  estimate  of  the  reliability  of 
Boca  Raton  and  Langley  Field  course  grades  used  in  the  first  valida¬ 
tion  study  depends  upon  how  representative  these  classes  were  of  the 
larger  group. 

The  reliability  of  classroom  grades  was  estimated  by  correlating 
the  average  grades  for  odd  weeks  with  the  average  grades  for  even 
weeks.4  Similarly,  for  flight  and  trainer  grades,  average  grades  on 
odd  missions  were  correlated  with  average  grades  on  even  missions. 
Odd  and  even  eonr.-e  grades  were  computed  by  averaging  the  odd  and 
even  ground,  trainer,  and  flight  grades.  Raw  flight  grades  were 
weighted  t'.O  percent  while  ground  and  trainer  raw  grades  were  each 
weighted  L'O  percent.  The  odd-even  correlations  and  the  correlations 
corrected  for  length  lire  presented  in  table  11.3. 

The  auxiliary  sco[>c  missions  referred  to  in  the  table  arc  flown  at 
the  auxiliary  PPI  scope  in  the  nose  of  the  aircraft.  On  such  missions 
the  student  operated  the  hombsight,  observed  the  auxiliary  scope, 
did  follow-thc-pilot  navigation,  and  maintained  a  navigation  log. 
This  initial  aerial  training,  to  which  0  to  10  flights  were  devoted, 

*  Sk'f.  John  S.  Hardlnx  planned  the  method  used  fo*  cHtlmntlntf  the  reliability  of  the 
couriKi  ternderf.  The  nlutln(lnil  work  win  Accomplished  by  S>;t.  Samuel  D.  Morford. 
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M.  \  ( il  to  f:i  111 1 1  i;i  i';z.'  tin'  -tndent  \s  i :  1 1  (In*  bomb  -ighl ,  i n t <  i  pn-l atiou 
,,f  iciiinis  on  1 1 1 •  *  I’l’I  scope,  and  coordinated  bombing  to  liiii<|ii(‘.  No 
ivhability  is  reported  for  bench  ret  trainer  grade-,  rim  e  only  tlie  filuil 
buirli  set  cheek  score  was  available  for  a  huge  portion  of  the  sample. 


Tuiur  1 1.3. — Kxt  iimit  in  of  rt  I  inhil  i  t  ]/  of  rndor  -ohm  m  r-ioumr  yrndi  h  mill  ruin - 
pom  ill  unuli  *  for  cliixarx  J.'i-ll  throwjh  J5-/.7,  I.mi'jh  y  I'iihl,  \'tiltdntion 
Study  l 


Oriole 


fo  1 


Clivmon*: 

t  operation . 

Nav  liution . 

llniril'InR.  . 

K  .'lur  Intelligence . 

Truiti'T 

T'OiiIc  . 

riu>e 

A  mil  iry  i-coix’  missions 

M-tm  m  u|K'  missions _ 

Cones’  grade . 


rn 

7:1 
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Art 


io  in 
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-  i  ft 
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i  Corrected  for  length  1'V  tl>"  S|x  nrumn-!!rown  | »rsj|>lu’cy  formula. 
'  S i r 1 1 f U n 1 1 1  nt  tin*  hjiercvnl  level, 
i  Mgmllmiit  nt  tlip  6-j*  ms  nl  level. 


If  this  estimate  is  representative,  the  course  grades  for  Boca  Baton 
and  Langley  Field  have  a  corrected  reliability  of  0.05.  To  the  extent 
that  the  odd  and  even  grades  in  any  component  of  the  aggregate  were 
not  independent,  i.  e.,  were  systemat  ically  biased  in  the  same  direction, 
this  figure  is  an  overestimate.  Because  the  grades  assigned  at  Victor¬ 
ville  are  composed  of  subjective  ratings  to  a  greater  extent,  their 
reliability  is  thought  to  he  somewhat  lower. 

Composition  of  Validation  Samples 
The  cases  used  in  computing  validity  coofiicieuts  were  selected  from 
the  1,330  students  in  classes  shown  in  table  11.1.  Only  radar  oWerver 
students  who  were  rated  bombardiers  or  navigators  were  u-ed.  A 
number  of  cases  were  eliminated  because  their  specialty  rating  was  not 
identified  or  because  they  were  eliminated  pilots.  Other  cases  were 
eliminated  because  of  incomplete  scores  for  the  November  10-13  air¬ 
crew  classification  battery.  The  samples  of  navigators  or  Kunbaruicrs 
v. ho  had  taken  any  other  single  battery  were  too  small  to  warrant 
statistical  analysis.  Many  other  cases  were  lost  because  they  ecu  hi 
not  lie  identified  on  the  course  grade  rosters  or  located  on  the  microfilm 
i'"i  tors  of  air-crew  classification  test  scores.® 

The  validation  samples  consisted  ot  ;»St»  cases,  fii>5  bombardiers  and 
•'Ll 

avigators.  The.  distribution  of  bombardiers  and  navigators  ;  y 
schools  is  presented  in  table  1 1 .1.  Comparing  the  cases  from 
Baton  and  Langley  Field  with  the  Victorville  cases  in  the  sample,  it  is 
to  be  noted  that  relatively  more  navigators  are  from  Victorville,  This 

‘  Cor:. |  te  B(»ta  0f  aircrew  cJamdflcnllou  test  acorr*  are  recorded  on  nVrof.ln  rv-Cr-r*  for 
(Ul  I.tIU  !  .m  Tm  who  took  niijr  of  the  luttlrrlc*.  Thcv>  r»>lem  a  tv  prepared  hj  ll»vw 
AAK  Training  Command. 
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difference  is  significant  at  tin:  1  percent  level.  Since,  as  already 
pointed  out.,  Victorville  course  grades  are  bn-ed  upon  standardized 
measures  to  a  lesser  extent  than  t hose  f mm  the  ot her  t  wo  schools,  there 
is  a  systematic  difference  between  bombardier  and  navigator  course 
grades.  .Specifically,  the  grades  for  bombardiers  are  ba.-ed  to  a  greater 
extent  upon  Radar  Project  tests  and  performance  cheeks  than  are  the 
grades  for  navigators.  This  difference  is  actually  greater  than  it  ap¬ 
pears  because  the  navigators  from  Boca  Raton  and  Langley  Field 
were  in  the  early  clas.-es  in  the  sample.  Because  more  tests  and  checks 
were  put  into  use  for  later  classes,  navigators  from  these  schools  had 
fewer  standardized  measures  contributing  to  their  course  grades  than 
did  bombardiers. 


Taiu.k  11,1. — V  til  idol  inn  cases  selected  from  1,339  students  in  consecutive  radar 
observer  dosses;  Validation  Study  I 


Validation  cases 

School 

All  cases 

—  .5; 

tl 

VI 

! 

Bombardier 

sr.mph 

Navigator 

sample 

I.anplcy  .  . 

400 

169 

52 

107 

li  icn  Union  . 

252 

S3 

62 

21 

Vhtorvlllo . 

.w; 

314 

91 

’  263 

Total . 

1,  339 

| 

6SG 

205 

3SI 

Validation  Statistics 

The  validity  coefficients  for  the  variables  in  this  study  are  presented 
in  table  11.5.°  They  will  he  interpreted  along  with  the  results  of  vali¬ 
dation  study  II  in  the  section  beginning  on  page  257  which  presents  the 
factors  indicated  to  have  appreciable  validity.  Examination  of  table 
11.5  shows  Mint  two  of  the  validity  coefficients  for  the  bombardier  sam¬ 
ple  are  significant  at  the  1-percent  level  while  an  additional  five  are 
significant  at  the  5-pevccnt  level.  For  the  navigator  sample,  five  co¬ 
efficients  are  significant  at  the  1-percent  level  and  an  additional  four 
are  significant  at  the  5-percent  level. 

A  multiple  correlation  was  not  computed  for  the  valid  variables  in 
this  study  because  the  same  tests  arc  included  along  with  a  number  of 
other  tests  in  validation  study  II  and  are  validated  against  what  is 
thought  to  he  a  better  criterion. 

Correction  for  restriction  of  range. — The  absolute  sizes  of  the  valid¬ 
ity  coefficients  must  he  evaluated  in  the  light  of  the  various  selection 
procedures  to  which  the  samples  arc  known  to  have  been  subjected. 
Four  major  selection  procedures  which  have  operated  to  restrict  the 
ranges  of  various  abilities  in  the  present  samples  are:  the  Aviation 

•  I’roivMilnR  of  these  validation  (lata  on  International  Business  Machine  punch  cards  was 
planned  and  earrled  out  hy  I.t.  Roshal  with  the  assistance  of  S/SkO  Johnston,  Sj;t.  William 
J.  Manpan,  S«t.  Harold  I.  Kausli.  nnd  S/Sgt.  Sullivan.  Sgt.  Ilium  computed  the  correla¬ 
tion  coetlldents  from  these  muchlne  data. 
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■  I’rodurt nwmrnt  c firn  lnlinitn  h<  lirrrn  thr  radar  obts'n  cr  courtc 
tjradr  and  Mitrtion  tariabh  .1  from  Validation  Study  l 
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1  In  the  c  c.  lumps  are  |>rv5»Tit«  «I  the  M's  ami  SICs  obtained  from  rv-! :i- ’ l* 1 1 «t i.-.s  lo  u ne‘.\  *  0-d  aviation 
rill  !■:!«.  U> Mills  t>f  t<  >!ir.e  M  |’>> ch'.|...*ird  Kev<  arch  and  M.dmJaud  r?>clud..f  ..•  .1  K.iun.lu.ni: 
I  r.:t *  arc  pn  <*  r  1 1 /*»  1  separately  Ivmuse  of  M.'utfir.ant  did.  ri*f»irs  10  the  *  I  1  d*:'  ■!  r tat isi..s.  It  "  as  r.  >t  cMdy 
f-v  t>  t:>  t>  riiiSnc  for  vahdrits.n  s.inipl.  s  »h:o  the  Indivld-.i  d«  wire  ytxcn  lltir  clssi.f  eats  o  lr<x. 
it  is  ll.oit.'ht  tint  tlo  i!‘.:i;..rity  of  th'tn  «<  re  t  d  at  t'syi  hi  !<  .  ..-al  It'  •<  .red)  l'  it  Its.  'I  ho  M.-li  -al  an  4 
l'«!  '  !  .il  F  sammi:.;:  t  'uit  d  it  a  are  ha**  d  ii|«>n  1>." cm.  «■  t«  - t •  •!  M  l’  i.:t»  4  to  in,  r<j».ri« <1  m  Itrv  .r.~h 
''  *411.  r  y  r  !.■  1  d  S. .  !i.;i.  other  of  tl  r  Sur,.-.  11.  1 1  <| .  A  A  I-  TO.  Fort  Worth,  r.  X  .  10  Jan  1944. 

'It.r  l‘>>  c !  oh  „■;{  1 1  Hi  .  arch  In  it  data  arc  h  tc-J  tif«m  t.,'*  *1  no  s  t<  •  vl  .at  f  ’  r,:t  *  1  to  A,  n  7  'Cel  l:i  U I  O'.'.rt'h 
N'c'o  '413.  l'-y  c  hoi",  1.  I  rift,  ill,  Ott.oc  of  the  Sur^.-on,  liij.  AAF  I  C,  Fort  Worth,  lit.,  I  Fob.  tl»44. 

1  r/  .1.1  at  the  1|.  runt  level. 

1  '  -•■■■<  1  t  .-.t  th«|  A  p.  n»  r.t  ],  vrl. 

*  "I  1  'it  ,!:•!.'<  arc  !■  .1  on  U7  bombard).  rs  olio  In.  I  taken  the  H'.'.ar  Perec  nine  Buttery. 

•  'I  I  <  -  • :  ,t;  s.i  a  are  I.  c. .1  upon  all  l.inb..rdi.  rs  (N-6.s|3)  who  t.K.k  tlx  Uadar  Ubv  mr  Si  l.cUon  Btl- 
t;ry  I  r  to  |  Apr.  IMA. 

1  I'  I  :  t  .1 1  i  t:<  <  are  hared  ijjsmall  iiavi„-j’.<.rs  (N-3.7333)  \s5iot.>ok  ttic  Usd.ar  Observer s<l.rtl.*n  Battery 
:  fi  r  to  I  Apr.  IMA. 

C:u]i't  Qualifying  Exa  ini  not  ion  administered  prior  (o  aviation  stu- 
‘lent  status,  the  air-crew  classification  battery,  proficiency  grad  s  in 
bombardier  or  navigator  training,  and  the  radar  stanino  and  other 
variables  described  in  chapter  10.  It  would  be  highly  de.-irnl  V  to 
be  able  to  compare  the  validities  obtained  in  this  study  with  those 
obtained  in  other  air-crew  specialties  for  unclassified  aviation  students. 
This  would  constitute  comparison  with  samples  that  were  not  re- 
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si  riel  oil  on  tin*  hasps  just  (Ipm  rihptl.  llmvevi-r,  this  comparison  could 
not  be  made  because  the  dal  a  required  for  satisfactorily  correctiim 
validities  for  such  a  complex  set  of  rivt  rifting  variables  were  not 
available.  To  enable  the  reader  to  make  his  own  estimates  of  the 
amount  of  restriction  that  occurred  in  the  various  test  scores  due  to 
the  restricting  influences,  means  and  standard  deviations  obtained 
from  administrations  of  the.  sell et ion  variables  to  samples  of  unclas¬ 
sified  cadets  are  presented  in  table  11.5  parallel  to  the  same  statistics 
for  the.  present  validation  samples.  The  reader  will  note  that  while 
most  of  the  standard  deviations  for  the  validation  samples  are  some¬ 
what  smaller  than  the  unrestricted  standard  deviations,  several  tests, 
notably  Reading  Comprehension,  Mathematics  A,  and  Mathematics 
B,  have  larger  standard  deviations.  These  increases  constitute  an 
artifact  which  arises  because  the  tests  in  question  are.  too  difficult  for 
the  general  aviation  student  group.  Scores  for  such  a  group  pile  up 
at  the  low  end  of  the  frequency  distribution.  After  selection  occurs 
and  this  pilo-up  of  low  scores  is  eliminated,  the  standard  deviation 
increases.  It  will  also  be  noted  that  the  unrestricted  means  for  tests 
which  were  weighted  in  computing  the  bombardier  or  navigator 
staninc  (see  appendix  A)  arc  lower  than  the  means  in  the  correspond¬ 
ing  bombardier  or  navigator  validation  sample. 

Formulas  arc  available  for  correcting  correlations  for  restriction  of 
rhnge  based  upon  a  single  variable.7  Using  these,  the  validity  coeffi¬ 
cient  obtained  for  137  bombardiers  who  had  taken  the  radar  observer 


*  Two  formulas  were  used  in  the  present  study.  Doth  were  derived  by  Knrl  Pearson 
(Mathematical  Contributions  to  t lie  Theory  of  Evolution — XI.  On  the  Influence  of  Natu¬ 
ral  Selection  on  the  Variability  and  Correlation  of  Organa.  Philosophical  Transactions  of . 
the  I  loyal  Socic'.y  of  Loiuton,  Series  A,  vol.  200  (March  1003),  pp.  1-00). 

Notation  : 

Si,  S:,  S3  —  standard  deviations  In  unrestricted  distributions  of  variables  1,  2,  anil  3. 

■'ii  s3~:  stand. ml  deviations  In  restricted  distributions  of  variables  1,  2,  and  3. 
fii:  =  correlations  between  variables  1  and  2,  neither  of  thorn  restricted. 
r tv,  I.-,  nirndat Iniiti  between  variables  1  and  2,  1  and  3,  2  and  3,  respectively,  either 
or  both  vat!  tides  restricted  directly  or  Indirectly. 

Formula  1  :  Uteil  to  correct  the  correlation  between  variables  1  and  2  when  the  restric¬ 
tion  Is  based  on  variable  1  and  the  ratio  of  the  unrestricted  to  the  restricted  standard 
deviations  of  variable  l  Is  known.  Tills  Is  used  In  the  present  study  to  correct  stanlne 
validities  since  the  restriction  Is  based  on  the  stanlne  and  the  ratio  of  the  unrestricted  and 
restricted  stanlne  standard  deviations  can  be  determined. 


/  Si* 

■V  ‘-'"Mm*’  -TfT 

Formula  2  :  Used  to  correct  the  correlation  between  variables  1  and  2  when  the  restric¬ 
tion  Is  based  upon  variable  3  and  lb?  ratio  of  the  unrestricted  to  the  restricted  standard 
deviations  of  variable  3  Ih  known.  This  Is  used  In  the  present  study  to  correct  the  correla¬ 
tion  between  n  test  and  criterion,  rt>.  knowing  the  correlation  between  test  and  stanlne, 
r  u,  the  correlation  between  criterion  and  stanlne,  r  Bp  and  the  ratio  of  the  restricted  and 
unrestricted  stanlne  standard  deviations. 
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cok'd  ion  battery  were  corrrctcil  for  rot  rid  ion  of  range  based  upon 
i  nlar  siniiinc.  These  corrected  coefficients  might  be  u  eful  in  provid¬ 
ing  better  est  imates  of  the  validity  of  tests  for  selecting  radar  observers 
from  among  graduating  bombardiers  and  navigators.  Correction  for 
range  restriction  might,  for  example,  change  the  relative  order  of  tho 
tests  in  terms  of  validity.  The  corrected  eoellicients  are  not  presented, 
however,  because  they  differ  less  than  0.01  from  the  uneorreeted  co- 
eftieients.  Even  though  a  radar  stauinc  of  f>  or  higher  was  required 
for  entrance  to  later  radar  observer  classes,  there  was  little  restriction 
of  radar  stauinc  variability  in  this  sample.  The  dandnrd  deviation  of 
radar  stanines  for  the  137  bombardiers  was  l.Cej,  only  slightly  smaller 
than  the  unrestricted  standard  deviation  of  radar  stanines  (1.70)  for 
all  bombardiers  who  took  the  radar  observer  selection  battery  up  to 
1  April  104o  (N---  0,813). 8  Furthermore,  the  correlations  of  radar 
btaiiine  with  the  criterion  (0.08)  and  with  the  selection  tests  other  than 
these  included  in  the  radar  observer  selection  battery  (median  r  — 0.00, 
ra.  go—  —0.11  to  0.27)  were  too  low  to  make  .he  test  validity  correc¬ 
tions  worth  while.  No  correction  for  restriction  due  to  selection  in 
radar  stanino  was  possible  for  navigators  since  no  navigator  in  this 

study  had  been  given  the  radar  observer  selection  battery. 

» 

VALIDATION  STUDY  II 

Validation  Study  II  consists  of  t lie  validation  of  experimental  selec¬ 
tion  tests  as  well  as  tests  from  the  air-crew  classification  and  radar 
observer  selection  batteries  described  in  Validation  Study  I..  The 
criteria  consist  of  radar  observer  course  grades  for  classes  graduating 
prior  to  or  during  the  week  of  the  cessation  of  hostilities  on  14  August 
’Ob’).  Tho  course  grade  criterion  for  classes  from  Victorville  is 
similar  to  that  used  in  Validation  Study  I.  Course  grades  for  Boca 
Raton  and  Langley  Field  were  computed  by  Psychological  Research 
Project  (Radar)  wholly  on  the  basis  of  scores  from  standardized  tests 
and  performance  checks.  An  additional  study  used  bombing  accuracy 
us  a  criterion  for  the  Victorville  sample. 

Variables 

This  study  validates  the  variables  in  three  batteries  of  selection 
tests:  tho  air-crew  classification  battery  (November  1043),  the  radar 
oh:  •erver  selection  battery,  and  an  experimental  battery.  Appendix 
A  describes  each  of  the  tests  of  which  the  three  batteries  arc  composed. 
The  report  of  Validation  Study  I  describes  the  content  of  tho  first 
two  batteries  and  their  use  in  classification  and  assignment. 

The  experimental  battery  consists  of  10  printed  tests,  0  psych omo- 
tor  tests,  and  a  second  indication  of  preference  for  radar  ol>  •erver 

*  Syt.  Allirrt  IVpIfonr  roniptitnl  the-  unr*\<trlctrcl  ni'-.tn*  runt  MamlniM  <IrvlnU"mt  <•(  tha 
rn,lir  Htnnlnc  for  boiul>ardlern  and  navigators.  Tlir*r  arr  pH-M-iitnl  In  t.iblo  11.5;  »*•« 
f'»'tnotcn  (5)  nnd  (0). 
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training,  Radar  Preference  IT.  The  battery  was  administered  by  tho 
Radar  Project  to  bombardiers  and  navigators  in  the  week  prior  to 
their  cut  ranee  into  radar-ob-erver  t  raining,  The  experiment  al  battery 
was  not  used  in  classification  or  assignment. 

In  all,  7*2  selection  variables  are  represented  in  Validation  Study  II. 
From  the  air-crew  classification  battery  and  the  radar  observer  se¬ 
lection  battery  come  the  same  27  variables  validated  in  Study  I.  From 
the  experimental  battery  there  are  15  variables,  since  for  the  19  printed 
tests,  both  right  and  wrongs  were  validated. 

The  experimental  battery. — It  will  be  remembered  from  chapter  3 
that  the  first  experimental  selection  battery  was  assembled  entirely 
from  tests  already  available.  Plans  at  that  time  called  for  the  valida¬ 
tion  of  additional  batteries  which,  is  was  expected,  would  include 
newly  developed  tests.  In  assembling  the  first  battery,  the  decision 
was  made  to  limit  it  primarily  to  tests  of  intellectual,  perceptual,  and 
motor  abilities,  reserving  the  major  part  of  the  projected  second  bat¬ 
tery  for  measures  of  interest,  personality,  background,  etc. 

Tho  choice  of  printed  tests  for  the  first  experimental  battery  was 
made  in  terms  of  tsvo  contrasting  approaches  to  selection  test  research. 
Those  approaches  are  described  in  chapter  4  as  the  factor  test  approach 
and  the  job  analogy  test  approach.  It  will  be  remembered  that  factor 
tests  are  those  tests  which  are  relatively  independent  of  other  measures. 
Individually  they  tend  to  have  low  validities,  but  because  of  their  low 
intercorrelations  they  may  in  combination  produce  high  validity.  It 
is  believed  by  some  psyehometriciuns  that  eventually  a  limited  number 
of  factor  tests  will  be  developed  with  which  it  will  be  possible  to  pre¬ 
dict  success  on  any  job  merely  by  differentially  weighting  tho  various 
scores. 

Job  analogy  tests,  on  the  other  hand,  individually  tend  to  have 
higher  validities  than  do  factor  tests.  They  also  usually  have  relatively 
high  intercorrelations  sinco  they  test  complex  functions.  Conse¬ 
quently,  a  combination  of  job  analogy  tests  often  yields  littlo  higher 
validity  than  the  most  valid  single  test. 

It  was  decided  to  choose  printed  tests  for  the  experimental  battery 
on  the  basis  of  both  the  factor  and  job  analogy  view.9  The  decision  to 
work  from  both  points  of  view  was  based  upon  two  considerations. 
First,  sinco  in  tho  present  stage  of  selection-test  research  neither 
approach  is  self-sufficient,  tho  use  of  cither  alono  was  not  justified. 
Second,  considerable  interest  was  felt  in  a  comparison  of  the  relative 
efficiency  of  the  two  approaches  at  their  current  level  of  development 

Psychomotor  tests  for  tho  experimental  battery  were  not  selected 
within  the  above  framework.  Their  choice  was  based  upon  an  inde- 

•  Ciipt.  I.loyd  G.  Humphreys  nuil  Sul.  Ilymnn  Heller  selected  these  tests. 
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pendent  job  do.-vription  of  the  radar  ok-ervor’s  t a>k  by  representati vos 
from  tin*  School  of  Aviation  Medicine.10 

Also  in  the  battery  was  Radar  Preference  IT,  an  indication  of  in¬ 
terest  in  radar  observer  training  made  on  the  Radar  Student  Informa¬ 
tion  Blank,  described  in  appendix  A. 

The  printed  and  psycliomotor  tests  constituting  the  experimental 
battery  are  listed  below'. 

Tests  chosen  as  factor  tests. — The  following  tests  were  included  in 
the  experimental  battery  because,  of  evidence  that  they  are  relatively 
pare  measures  of  factors  that  have  been  definitely  or  tentatively  found 
in  analyses  of  test  batteries  in  the  Aviation  Psychology  Program.11 
Each  test  is  presented  with  the  factor  it  measures  and  the  loading  it 
lias  in  that  factor.12 

Aerial  Orientation,  CP520A,  is  thought  to  bo  a  relatively  puro 
measure  of  the  space  I  or  spatial  relations  factor.  Loading  is  unknown. 

Area  Visualization,  CP815A,  is  thought  to  measure  the  visualization 
factor  with  a  loading  of  0.50. 

Air  Corps  Vocabulary  (1012)  has  a  loading  of  approximately  0.70 
in  the  verbal  comprehension  factor. 

Compass  Orientation,  CIGGOA,  is  thought  to  represent  a  hypotheti¬ 
cal  factor  by  the  same  name.  Loading  is  unknown. 

Estimation  <  f  Length,  CPG31A,  represents  a  postulated  length  esti¬ 
mation  factor.  Loading  is  unknown. 

Flight  Orientation,  CP52SA,  is  thought  to  bo  a  relatively  puro 
measure  of  the  space  I  or  spatial  relations  factor,  but  its  loading  is 
unknown. 

Mechanical  Information,  CI905B,  measures  the  mechanical  expe¬ 
rience  factor,  with  a  loading  of  0.75. 

Memory  for  Landmarks,  CI510AX2,  measures  the  memory  I  or 
rote  memory  factor.  An  almost  identical  form,  CI510AX1,  has  a  load¬ 
ing  of  0.G0  in  this  factor. 

Numerical  Operations,  CI702BX1,  represents  the  numerical  facility 
factor.  An  older  form  of  this  test,  CI702A  and  B,  has  a  loading  of 
0.80. 

Pattern  Comprehension,  CPS03A,  has  a  loading  of  approximately 
0.50  on  the  visualization  factor. 

'*T1i1h  Jolt  denerlptlon  wn*  carried  out  by  Cnpt.  Judaon  Drown  nrnl  Capt.  Oli-nn  Finch. 

11  Of  the  t oh t h  In  t lio  November  191.1  Air-Crew  Cln«Hlflcntlon  llattery,  Instrument  Com* 
prehension  II,  Cnilflll,  wn*  connldored  to  be  n  factor  tent.  It  In  probably  the  1>«  it  ;  .  1  tlua 
r-eaHurp  „f  Hpnce  I  with  n  loading  of  approximately  0.50.  Severn!  fnclnrit  Li.-.wii  to  t>« 
inenHnreil  more  or  1>ki  adequately  t>y  tej-tn  In  the  claH'dflc.utlon  buttery  were  not  imunureil 
f'7  tc-htu  tiii-lnibil  In  the  experimental  buttery.  Thene  Include  pilot  Interrai,  measured  by 
Ui-nernl  Information,  <'K."d)5K  ;  pHyehomotor  coordination.  mrarnred  by  Complex  IVordliia* 
"on,  CM701A,  and  ltotary  l’urmilt.  Cl’41011;  nnd  payi-lioniotnr  piwlelon,  wrnuurvd  by 
I'lnei-r  Dexterity,  CM1H5A,  and  Dlnerlmlnatlon  Heart  Inn  Time,  CI*0I11>. 

uTh«*  factor  loading*  preo-nted  In  thin  motion  are  taken  prbnnrlly  from  nunl/ec*  of  th* 
July  10^3  imd  November  1013  air  crew  claaaltlrntlon  Iinttrrlc*  reported  by  l*»y chologtcal 
tt'  v.urxh  Unit  No.  3. 
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Position  Orientation,  CP.VJUA,  iepre-enls  a  j *< i - 1 1 1 1  :i t < •«  1  spin-o  II  or 
rot  ;it  ioiial  space  factor.  An  earlier  ver.-ion,  Hands,  CP.M2A,  lias  a 
loading  of  0.15  on  tliis  factor. 

Spatial  Reasoning,  Cl  21  1  BX2,  meaMiiv.s  t lit*  reasoning  I  or  general 
reasoning  factor  with  a  loading  of  t)..rif>. 

n  r~\ 

Speed  of  1  ( lei  1 1  dicat  ion,  CPtllOC.  repre.-ents  the  perceptual  speed 
factor.  An  older  form  of  this  test,  CPOIOB,  has  a  loading  of  approxi¬ 
mately  0.(>5  on  this  factor. 

Visual  Mi  mory,  C  la  1  I  A,  is  thought  to  measure  a  pu-t  uhitcd  factor 
called  memory  II  or  visual  memory.  A  roughly  similar  test,  Map 
Memory,  Cl. ’>05  AX  2,  which  contains  line  drawings  instead  of  photo, 
graphs,  lias  a  loading  of  0.00  in  memory  If. 

Testa  chosen  as  job  analogy  testa. — The  following  tests  were  included 
in  the  experimental  battery  on  the  basis  of  similarity  to  a  task  carried 
out  by  the  radar  observer.'1 

Pattern  Identification,  CPS20A,  developed  by  the  Air  Crew  Evalu¬ 
ation  and  Research  Detachment  No.  1,  resembles  the  observer’s  task  of 
identifying  patterns  of  returns  on  the  PPI  scope. 

Polar  drill  Coordinate,  CPStbB,  presents  the  subject  with  tasks 
similar  to  those  involved  in  '-ending  fix  data  from  the  PPI  scope  and 
plot  t  ing  it  on  a  limp. 

Seale  Reading,  CPthiTA,  requires  reading  scales  that  are  very  simi¬ 
lar  to  the  numerous  scales  read  by  the  radar  observer  in  carrying  out 
bis  navigation  and  bombing  tasks. 

Spot  Location,  CPS18A,  presents  a  task  which  is  analogous  to  the 
radar  observer’s  tusk  of  quickly  finding  points  in  n  pattern  on  the  map 
which  lie  sees  in  a  similar  pattern  on  the  PPI  scope. 

Numerical  Operations,  CI7U2BX1,  included  in  the  experimental 
battery  on  the  basis  of  relatively  pure  factor  content,  was  also  con¬ 
sidered  to  be  u  job-analogy  test.  It  requires  simple  arithmetic  calcu¬ 
lations  similar  to  those  carried  out  by  the  radar  observer  in  navigation 
and  bombing. 

Ratio  Estimation,  CP225A,'  was  included  in  the  experimental 
battery  even  though  it  is  not  clearly  analogous  to  a  task  of  the  radar 

"Of  tire  testa  In  t lie  radar-observer  eelectlon  bntlory,  three  were  considered  to  be  Job- 
Analogy  tenia. 

Oi  ion!  I  mile  Itc.idlne.  (’l’J-’I  A  nr  U.  require*  oiierntlun*  Identical  with  rending  tbe  rang* 
nml  bearing  of  n  target  on  the  I'l’I  neope. 

On'll|nHrn|io  Interpretation.  CUS17A,  Involves  pcrreplunl  tasks  nlrnllnr  to  detectlnf 
returns  on  the  1*1*1  nml  A  scopes. 

I’nllern  Orientation.  CPHIrtA.  requires  the  Identlfloutlon  of  rotated  pnttrrnB  which  H 
nnnlngoiiM  to  reeo-inlrliig  putteriiH  of  returns  on  the  I'l’l  scope  when  nrJmuth  stabilization 
Is  not  f uiict lou lug  on  the  rndiir  set 

Of  the  tests  In  the  alr-crew-olasslfleatlou  battery,  two  were  considered  to  be  Job-analogy 
tests.  1  Uni  and  Table  Hemline.  CI’lijeA  and  CI’tl2IA.  was  thought  to  be  analogous  to  tbe 
rndiir  observer's  dial,  sente,  and  table  reading  tasks.  Spatial  Orientation  I,  CP501B, 
seemed  to  present  u  protdem  blmllnr  to  that  of  Identifying  patterns  of  scope  returns  on* 
target  photo  and  vice  versa. 
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observer.  It  is  not  included  in  the  comparison  of  the  factor  and  job- 
analogy  approaches  to  he  presented  below. 

Pwjcho  mot  or  Texts. — The  following  psychomotor  tests  were  in¬ 
cluded  in  the  experimental  battery  on  the  basis  of  the  recommendation 
of  the  School  of  Aviation  Medicine: 

Check  List  Dial  Setting,  Model  A  (no  code  number). 

Complex  Coordination,  CM701E. 

Kate  Control  Test,  CMS'iuA. 

Self-Pacing  Discrimination  Reaction  Time,  CPtillE  modified. 

Thurstone  Two-Hand  Pursuit,  CMSIOA. 

Visual  Coincidence,  CP(>p}B3  modified. 

This  battery  was  administered  only  to  radar-observer  students  at 
Langley  Field. 

Plans  for  further  experimental  batteries. — As  pointed  out  above,  no 
measures  of  personality,  attitudes,  or  interests  were  included  in  tho 
experimental  battery.  It  was  planned  to  validate  several  successive 
batteries  with  increasing  emphasis  upon  such  measures.  .  The  specific 
tests  which  were  planned  for  inclusion  in  a  second  experimental  vali¬ 
dation  battery  14  are  us  follows: 

Indices  of  Self-Confidence,  CE127E,  is  w  self-rating  scale  of 
psychomotor  test  performance. 

Technical  Information,  CE509A,  is  a  collection  of  five  sports  and 
hobbies  tests  including  items  on  hunting,  firearms,  radio,  photog¬ 
raphy,  and  electricity.  This  test  was  to  be  included  on  the  hypothesis 
that  amount  of  information  of  the  typo  asked  for  is  indicative  of 
strength  of  interest. 

Satisfactions  Test,  CE-100C,  is  concerned  with  the  subject’s  likes  and 
dislikes. 

Biographical  Data  Blank,  CEtiOTB,  consists  of  personal  data  items 
taken  from  the  CEfiOtiE  and  CEfiO-JF  forms  of  the  Biographical  Data 
Blank. 

The  Ilnmm-Wndsworth  test  measures  personality  traits  by  means 
of  subjective  questions.  It  was  planned  to  validate  the  epileptoid, 
liysteroid,  and  autistic  scales. 

Directional  Orientation,  CP')  15 1)  and  E,  measures  ability  to  main¬ 
tain  orientation  to  compass  directions. 

Camouflaged  Figures,  C'P^Ol  A,  requires  the  subject  to  distinguish 
patterns  from  confused  backgrounds. 

Object  Completion,  CPSllA,  is  constructed  to  measure  ability  to 
perceive  the  form  of  objects  when  only  portions  of  their  elements 
are  seen. 

“  H"t.  Hnnllntc  nnd  SK't.  Ilrllrr  were  erlninrlljr  r>'i<]>on»Uilf  for  (dunning  the  «o<-ond  rxp«rl« 
tn>'Mlnl  vnlldntlon  Imltcrj. 
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lVm  trat ion  of  Camouflage,  CPS17A.  very  similar  to  (.'ainonflnm'd 
Figure,  requires  the  detection  of  patterns  concealed  in  confusing 
backgrounds. 

Validation  Criteria 

HonibiinUcr  sant/ile:  Course  (jrndcs. — The  bombardier  sample  con¬ 
sists  of  students  front  Boca  Baton  and  Langley  Field.  The  radar 
observer  course  grades  forming  the  criterion  for  this  sample  were 
computed  by  the  Radar  Project,  entirely  from  scores  on  standardized 
t est s  and  performance  checks.16  For  the  Langley  Field  students, 

:  cores  from  a  complete  battery  of  proficiency  measures  constructed 
by  the  project  were  available.  A  complete  set  of  tests  and  chocks  was 
also  available  for  the  Boca  Baton  bombardier  .students,  but  all  checks 
containing  navigation  items  bad  been  modified  by  personnel  at  the 
training  station.  This  modification  relieved  the  radar  observer  of 
many  of  bis  navigational  tasks,  limiting  his  work  primarily  to  taking 
radar  fixes. 

Before  the  various  test  and  performance  check  scores  were  weighted 
and  combined,  their  raw  scores  were  converted  into  stanino  scores 
by  the  following  steps.  For  each  form  of  every  test  and  check,  a  dis¬ 
tribution  of  raw  scores  was  made  separately  for  each  school.  These 
raw  scores  were  then  converted  to  single-digit  scores  on  a  one-to-nine 
scale.  The  conversion  was  made  on  a  percentage  basis  assigning  a 
score  of  9  to  the  highest  4  percent,  8  to  the  next  highest  7  percent,  7 
to  the  next  12  percent,  G  to  the  next  17  percent,  5  to  the  middle  20 
percent,  1  to  the  next  lower  17  percent,  3  to  the  next  12  percent,  2  to 
the  next  7  percent  and  1  to  the  lowest  1  percent.  For  distributions 
where  it  was  possible  closely  to  approximate  those  percentages,  the 
conversion  yielded  a  distribution  with  a  mean  of  5.00  and  standard 
deviation  of  2.00. 

The  10  converted  test  and  performance  check  scores  for  each  stu¬ 
dent  were  combined  into  three  part -course  grades  which,  in  turn,  were 
combined  into  an  over-all  course  grade.  A  flight  grade  was  computed 
by  determining  the  average  of  the  final  aerial  check  score  given  a 
weight  of  2  and  the  intermediate  aerial  check  score  given  a  weight  of 
1.  A  trainer  grade  was  computed  by  finding  the  average  of  the  final 
supersonic  check  score,  given  a  weight  of  3,  and  the  radar  navigation 
intermediate  supersonic,  radar  bombing  intermediate  supersonic,  and 
final  bench  set  check  scores,  each  given  a  weight  of  1.  A  ground  grade 
consisted  of  the  average  of  the  final  test  scoro  (total  of  Final  Test  I 
and  II),  given  n  weight  of  3,  and  tho  radar  navigation,  radar  bomb¬ 
ing,  and  set  operation  intermediate  test  scores,  each  given  a  weight  of  1. 

"  S^t.  Heller,  wllli  tho  Assistance  of  S/Sct.  William  J.  Woyivod,  supervised  the  compo- 
tut Imi  of  course  (trades  from  the  l.ni'cley  Field  dntn.  Opl.  Robert  J.  rntterson,  with  the 
assistance  of  Cpl.  Irving  Fudomnn  and  Cpl.  Jnmea  C.  llolt,  did  the  snnie  work  on  tho  Doc* 
Union  data. 
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I'lic  part -course  scores  were  combined  into  an  over-all  course  grade 
1 1 v  computing  the  average  of  the  (light  grades,  given  a  weight  of  tj,  the 
trainer  grade,  given  a  weight  of  2,  and  the  ground  grade,  given  a 
weight  of  2.  In  computing  the  validity  statistics  course  grades  for 
Langley  Field  and  Boca  Baton  students  were  grouped  into  a  single 
bombardier  sample  of  tig',)  cases.  The  distribution  ,>f  course  grades 
for  this  sample  had  a  mean  of  1.01  and  a  standard  deviation  of  1,17. 

liombardier  Sample:  Reliability  of  Course  Crudes. — Because  no 
method  was  available  for  accurately  determining  the  reliability  of  the 
course  grades  computed  for  the  students  at  Langley  Field  and  Boca 
Baton,  an  attempt  was  made  to  estimate  the  reliability.  It  will  bo 
noted,  from  the  weights  given  the  various  tests,  checks,  and  part- 
course  grades  in  computing  the  final  course  grade,  that  the  final  aerial 
check  contributes  -10  percent  of  the  course  grade,  the  intermediate 
aerial  check  contributes  20  percent,  and  final  supersonic  check  and 
final  test  each  contribute  10  percent,  and  the  remaining  tests  and 
checks  each  contribute  3.33  percent.  To  estimate  the  reliability  of 
the  course  grade,  two  scores  were  obtained  for  each  student.  Knell 
part  score  was  based  upon  test  and  check  scores  which  together 
mado  up  50  percent  of  the  course  grade.  Also,  the  total  test  contri¬ 
bution  and  total  cheek  contribution  was  divided  equally  between 
the  two  part  scores.  On  this  basis,  score  “A”  for  each  individual 
consisted  of  the  final  aerial  check  score  weighted  by  10  and  the  three 
intermediate  test  scores  each  weighted  by  3.33.  Score  “B”  consisted 
of  the  intermediate  aerial  cheek  score  weighted  by  20,  the  final  super¬ 
sonic  check  score  weighted  by  10,  the  final  test  score  weighted  by  10, 
and  the  two  intermediate  supersonic  check  scores  and  single  bench 
set  check  score,  each  weighted  by  3.33.1* 

Only  those  students  were  included  in  the  sample  who  had  all  test 
and  check  scores.  For  the  Langley  Field  group,  the  bench  set  check 
scores  for  two  radar  sets,  the  AX/APS-15  and  AN/APS-15A,  wore 
averaged  and  treated  ns  u  single  score.  The  correlations  between  “A” 
and  “B”  scores  were  computed  for  the  Langley  Field  and  Boon  Baton 
groups  separately. 

The  correlations  between  the  “A”’  and  “B”  scores  for  278  students 
from  Langley  Field  was  0.27.  When  corrected  for  double  length,  tho 
coellieient  become  0.13  which  is  probably  a  better  estimate  of  eourso 
grade  reliability  than  the  nncorrected  part-grade  intercorrelution. 
Corresponding  coefficients  obtained  from  117  Boca  Baton  stud-mts 
were  0.23,  nncorrected,  and  0.33,  corrected, 

Navigator  Sample:  Course  Crude*. — Tho  navigator  sample  was 
composed  of  radar  observer  students  from  Victorville.  The  courso 
grades  used  in  this  validation  .-.w.dy  ..ere  those  assigned  by  tho  train- 

Tills  work  was  done  by  Cpl»  Fwlt*ninn  and  Cpl.  Owtn  It.  Mungrr. 
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ill"  s(:i( ion  since  too  few  test  ami  performance  check  scores  were  avail- 
able  to  form  by  themselves  an  adequate  basis  for  computing  course 
glades.  Practically  all  (lie  classes  used  in  this  study  bad  taken  t lie 
filial  and  three  intermediate  tests.  However,  the  only  performance 
check  scores  available  were  for  the  bench  set  and  final  aerial  checks, 
these  being  given  to  half  the  classes  Used.  Consequently,  the  course 
grades  used  for  the  navigator  sample  are  based  primarily  upon  in¬ 
structor  ratings  and  informal  written  tests.  Aerial,  ground,  and 
trainer  grades  were  weighted  by  the  training  station  in  the  ratio  of 
(i :  2  :  2  in  conipnt ing  the  over-all  course  grade. 

The  aggregate  scores  obtained  from  the  training  station  were  con¬ 
verted  to  standard  scores  with  a  mean  of  50  and  standard  deviation  of 
10.17  Two  such  conversions  were  necessary,  one  for  classes  -15-27 
through  -15-29  and  another  for  classes  -15-30  through  -15-33.  This 
was  necessitated  by  a  shift  in  the  grading  system  from  class  -15-29  to 
class  -15-30.  Xo  data  are  available  for  computing  the  reliability  of 
the  navigator  sample  course  grades.  The  distribution  of  220  con¬ 
verted  course  grades  for  the  navigator  sample  had  a  mean  of  50.58  and 
a  standard  deviation  of  9.94. 

iVariyator  Sample :  Circular  Error. — Records  of  bombing  accuracy 
were  available  for  the  navigator  sample  and  were  used  as  a  validation 
critcrian.  The  records  were  based  upon  camera  bombing  and  sum¬ 
marize  all  drops  that  were  photographed  and  scoreable.  The  dnta 
are  analyzed  from  the  point  of  view  of  reliability  ns  developed  in 
chapter  '9.  Various  information  important  to  an  understanding  of  the 
characteristics  of  the  circular  error  data  used  in  this  study  and  the 
conditions  under  which  they  were  obtained  is  presented  in  table  11.0. 
It  will  he  noted  that  there  is  great  variability  in  the  number  of  scored 
drops  from  which  the  average  circular  error  is  computed.  The  range 
of  from  1  to  19  drops  suggests  the  possibility  that  differences  among 
students  in  averago  circular  error  may  he  due  to  their  having  had 
different  amounts  of  practice.  If  this  were  true  it  would  tend  to 

Taiii.k  11.(1. — Distribution  statistics  for  circular  error  data  and  data  drscriplivt 

of  tin  conditions  u.idtr  idiieh  circular  error  data  were  obtained;  navigator 

sample,  Victorville,  Camera  /tombing,  Validation  Study  II 


Puts 

Mean 

Median 

SD 

Range 

Number 

of 

students 

A  vi  rare  circular  error  In  fret . . 

4.350 

4,000 

1,700 

1,400  to  U.300 

225 

Nunilvr  of  drops  M  ured  p.  r  student . 

0.  1 

1  to  10 

215 

Nmiilvr  of  ml-  lulls  ivr  -tiidci:i  from  which 

243 

so  red  drop,  acre  obtained . 

1 1 

4.1 

1.4 

1  too 

niiUii'-d  nltlludr  froiii  which  each  student 

343 

made  a  so-ri  d  drop. . . .  . . . 

tS,  808 

1,070 

11,000  to  10,800 

LoWi  si  altitude  from  which  each  student 

342 

made  a  m-ur.-d  drop  .  .  .  . 

It.  US 

10,  073 

l,  570 

7,260  to  15,000 

Nil  ml-  r  of  dill- lent  inr>.vlsupoii  which  each 

245 

si  udetit  made  scwn-d  drops . _ . 

5.0 

6.3 

CO 

1  to  11 

,r  S/Sgt.  Richard  T.  Mitchell  with  the  assistance  of  CpL  James  C.  McClure,  Jr.,  carried 
out  this  statistical  work. 


ma-k  :i n v  ifliit ionship  otherwise  cxi.-t ing  between  circular  error  and 
.-election  test  -cores.  To  determine  whether  practice  lending  to  im¬ 
provement  in  circular  error  \v;is  indicated  by  the  number  of  .'-cored 
diiips,  tin1  correlation  was  computed  between  average  circular  error 
r.nd  total  number  of  bombing  runs,  both  scored  and  uuscored.  The 
i-oellicicnt  obtained,  based  on  •_!!()  cases,  was  found  to  be  only  0.0 1 . 

'I'ho  evidence  presented  in  chapter  0  indicates  the  reliability  of 
average  circular  error  to  be  approximately  0.20.  Such  an  estimate 
is  not  inconsistent  with  a  correlation  of  0.20  found  between  circular 
error  and  course  grades.  This  eoeflieient  is  based  upon  the  22G  stu¬ 
dents  in  the  navigator  sample  and  is  significant  at  the  1  percent 
level. 

Composition  of  Validation  Samples 

Only  those  radar  observer  students  were  included  in  the  validation 
.samples  who  were  either  rated  navigators  or  bombardiers  and  who  had 
taken  all  of  their  proficiency  tests  before  11  August  1015.  Only 
those  classes  were  used  that  had  been  given  the  experimental  selection 
battery  on  entering  radar  observer  training.  The  radar  observer 
classes  used  from  each  training  station  are  listed  together  with  their 
graduation  dates  in  table  11.7.  Only  bombardier  students  were  used 
from  Langley  Field  and  Boca  Raton  and  only  navigators  from 
Victorville.  These  restrictions  left  approximately  1,100  eases.  This 
number  was  further  reduced  by  20  percent,  half  being  eliminatcu  be¬ 
cause  they  had  taken  air-crew  classification  bit tt cries  other  than  the 
November  1!)1‘>  battery  and  the  remainder  because  they  did  not  have 
both  criterion  data  and  radar  stanines.  This  final  restriction  was 
made  necessary  by  the  time  limitations  under  which  the  validation 
statistics  were  computed.  Since  each  of  the  72  selection  variables 
was  correlated  with  the  criterion  data  and  radar  stanine,  a  great 
saving  of  time  was  accomplished  in  the  International  Business  Ma¬ 
chine  sorting  and  tabulating  work  by  using  a  single  sample  for  the 
eom-sc  grade  and  radar  stanine  correlations  in  t he  case  of  the  bom¬ 
bardiers  and  a  single  sample  for  the  course  grade,  circular  error,  and 
radar  stanine  correlations  in  the  case  of  the  navigators.  The  filial 


Taiilk  11,7. — Radar  observer  classes  used  from  each  training  station  in 

Validation  Rtudy  II 


Training  atntlon 

Class.* 

OnvluiHon  ilstt* 

la.-K’ry . 

!*'•  1  Union 
^  i^.orviUe  ... 

4S-19  Ihroiiglt  <’>-3.1 . 

33S  through  MS  . 

<5- 27  llin.iirh  <  V33  . 

12  May  I'/IJ  K  Auzu.t  IMS. 

2-;  May  KU'-M  A';»i:.t  I'.SJ. 
r  July  navi'*  Aiirm  ms. 

- - 

215 


.samples  consisted  of  0:29  bombardiers  and  220  navigators.  It  is  jm. 
porting  to  note  that  paralleling  the  dilferenee  between  bombardiers 
and  navigators  is  a  difference  in  kind  of  criterion  used  in  the  vali¬ 
dation.  '{'lie  bombardiers’  course  grades  were  computed  by  the  Ihnlar 
Project  wholly  on  the  basis  of  test  and  check  scores.  The  navigators 
were  assigned  grades  by  the  Victorville  training  authorities  using  the 
few  available  test  and  check  scores  to  an  unknown  extent. 

Validation  Statistics 

The  results  of  the  validation  against  course  grades  for  the  bom¬ 
bardier  sample  are  presented  in  table  11. 8. 18 


T.um.k  1 1.8.—  Prod m  t-nunnent  corrrlu t ion s  bi  t  wren  the  rn<lor  observer  courts 
l/rriite  and  st  leelion  variables  for  the  bombardier  sample  in  Validation  Study  // 


Variable 

rn  1 

rn  ’ 

.rn* 

.V 

M  can  1 

„SD« 

Stanlncs: 

llomliiirdlcr  stanlne... . 

‘0.13 

0.  IS 

0  14 

C2S 

7.  14 

1.22 

Navigator  stanlno.. .  . 

A  14 

.23 

.  10 

1,29 

0.43 

1.35 

l’llot  st.mlne  . 

A  03 

.  17 

.09 

029 

5.37 

1.00 

ltadar  ohser\i-r  selection  battery: 

Coordinate  reading,  C  P22IA  or  B  . 

A  03 

.77 

.11  ' 

029 

117.09 

15.  fj 

(bcilln  t  ope  interpretation.  CI’SITA . 

.01 

.33 

.01 

029 

27.03 

Ass 

1  ’.it t <  ro  orient  it  bin,  CPMOA  .  . 

.04 

.  7.2 

.07 

029 

30.  79 

7.55 

Sp  it  d  orientation  1,  C  P501B . 

.07 

.45 

.09 

029 

33.  35 

4.55 

Ita  :  .•  preference  1 . 

.00 

-.04 

.00 

029 

8.  27 

1.20 

liad  .r  Manlno  . 

.07 

.  10 

029 

6.02 

1.1) 

).:•>. 

Air-crew  i  la.-  ideation  hntlery  (printed  tests): 

Bhigraphical  data  blank,  navigator  scoro, 
CEur.’l)  . 

.03 

.01 

.01 

029 

22.34 

Bln-graphlcal  data  blank,  pilot  score,  CK<V)2U _ 

-.07) 

.0-1 

-.01 

023 

26.59 

6  ;:t 

ln.il  and  table  re  tdtng.  CPr,22A  and  CPC2IA . 

>.  10 

.20 

.  12 

029 

■10.18 

6.9,' 

(leneral  Inform  at  Ion,  CKaOSK . 

•.  10 

.07 

.  10 

029 

39. 14 

13.TJ 

Instrument  comprehension  1.  C10ISB . 

-.02 

10 

-.03 

023 

9.39 

2  61 

Instrument  comprehension  11,  C10I6B . 

.02 

.20 

.01 

029 

31.04 

10  34 

Milt  1  n  iii.it  lea  A,  C1702F . 

'.  10 

.  12 

.  10 

023 

10. 10 

8.3 

M albeniatles  II,  < 1 12U0C . 

A  14 

.12 

.  14 

029 

10.18 

9.  it 

Meeleml.  d  principles,  C1‘!XJ3B . 

.01 

■  Cl 

.01 

029 

32  33 

9.  (0 

Keeling  comprehension.  (’101411 . 

A  10 

.02 

.  10 

029 

21.33 

12  94 

Spat  1  il  orientation  1.  CP6UIB . 

.00 

.29 

.03 

029 

31.94 

521 

Spatial  orientation  11.  (' P.VjJ  11 . 

.01 

.25 

.05 

029 

22.81 

5  34 

Alrcrt'W  Cl\< -.iflr.ii ion  ll.ittrry  (Psydiomotor  tests) 

< *< tin  1  •' t* \  ( * onnllu.it Ion,  C’MTOIA . . 

.07 

.11 

.03 

029 

64.  59 

9.3) 

1)1  i  it  in  mi  Ion  Ite.ii  t  ion  Time,  CI’CIID . 

All 

.03 

.  11 

029 

50.00 

5  74 

l-'ir. r  1  JeUerll  y,  <  'M  1  li>A . 

-.IV) 

.01 

-.00 

029 

00.22 

8.W 

Hot  if y  Pur-nit,  <*1*1101* . 

-.00 

.09 

-.05 

029 

53. 40 

10  (0 

Kildd.  r  Coni  ml,  CM  l’Alll  . 

A  os 

.02 

.03 

029 

70.23 

9.(0 

Two- Ham!  Ci  lord  in  it  ion,  C  M101 A . 

,00 

.02 

.00 

029 

62  52 

9. 76 

Kipei  iim  ill’ll  Printed  Tests: 

Acrl  d  Orient  at  Ion,  CP.')20: 

It [gilts  . 

A  12 

.13 

.  13 

4745 

22  70 

4.33 

Wrongs  . . 

-.09 

-.04 

-.09 

45(5 

7.  97 

180 

Area  VI  1  dl/utlun,  CP3I5A: 

lOght.s  . 

.02 

.10 

.03 

410 

34. 15 

5  97 

Wrongs  . 

.03 

.05 

,03 

410 

12  87 

53 

Air  Corps  Vocabulary  (1012): 

Kigfits  . 

.01 

.07 

.05 

450 

57. 90 

20.41 

Wrongs  .  . 

.01 

.  12 

.05 

450 

26.77 

11.06 

Comp.i  s  Orientation,  C1600A: 

Klght  1 . 

A  10 

.10 

.10 

450 

8&S5 

31.  SI 

Wrongs . 

-.02 

-.  17 

-.04 

450 

4.95 

iai» 

Estimation  o(  Length,  CP631A: 

Rights  . 

•  OS 

.04 

.09 

359 

29.82 

500 

Wrongs . 

.09 

.02 

.09 

359 

23. 14 

9.06 

Flight  Orientation,  CP528A: 

Rights  . 

.07 

.11 

.0-3 

457 

38.00 

5M 

Wrongs . 

-.03 

-.03 

-.03 

457 

8.72 

53 

See  footnotes  at  end  of  table. 

>•  l’riices-diig  of  these  validation  (lata  on  International  RutdncpH  Mnchlne  punch  cards  vrtt 
planned  and  carried  out  by  Lt.  Howlinl  with  the  n.salelnncc  of  H/Sgt.  Jolinaton,  Sgt.  Mnngao. 
Kgt.  Sheldon  II.  Nerby,  and  CpI.  Wilbert  It.  Sclnvotzer.  Tho  correlation  coefficients  \rtr* 
computed  liy  e'pl.  Arlene  E.  Bubcock,  Sgt.  Ilium,  S/Sgt.  Johnston,  Sgt.  Nerby,  Sgt.  It*®stU 
und  CpI.  Sehwotxer. 


Xuuj:  11  s’ — Prniluetitiniiient  correlation*  In  tier  i  the  radar  observer  course 
grade,  and  selection  variables  Jur  the  bombardier  sample  in  Validation  Study 
//—Continued 


Variable 

T\l  1 

hi* 

S\l  1 

D 

Mean  * 

sn » 

y  wrltii.  ntal  PUnted  Tests -Continued 
’  '  Mo.-!,  .i.ititl  Information,  C‘1W)5U: 

logins . 

0.  w 

0.  05 

a  oi 

in 

15  02 

AM 

Wrongs  . . . 

-.01 

-.05 

-.02 

441 

lilt 

A  10 

Memory  (or  Land  marks,  CI510AX2: 

Hlgnt.1 .  . 

.10 

.17 

.  n 

Kgl 

22  70 

7.57 

V  rones  .  . . . 

-.on 

-.03 

-.08 

KS1 

S.  3  2 

A  53 

N'un’.iTiial  Operations,  CI702BX1: 

llights . 

*.  PI 

.28 

.18 

in 

■oral 

8  01 

Wrongs  . . 

-.03 

-.02 

-.04 

413 

■rim 

3  13 

Pattern  Comprehension,  CPS03A: 

It  ip  1 1 ts . 

.03 

.31 

.01 

407 

18.  20 

ft.  83 

Wrongs  .  . . . .  . 

-.01 

-.05 

-.01 

407 

8. 13 

5.67 

I’attcr n  Identification,  CP820A: 

It  Ip  fi  ts  . 

.07 

.23 

.00 

407 

28.  M 

11.23 

Wroncs  .  . 

»-.  11 

-.10 

-.  12 

407 

10.  70 

7  .  33 

Polar  Orid  Coordinate,  Cl’SlOB: 

UiKlltS . 

>.10 

.12 

.  12 

432 

52.  35 

1X40 

Wroncs  . 

.03 

.  18 

.03 

433 

Kim 

883 

Position  Orientation,  CP520A: 

Bights . 

.08 

.32 

.  10 

45* 

211.  25) 

47.27 

Wrongs . . 

-.03 

.03 

-.02 

458 

12.  n 

10.78 

IUtlo  Kslimatlon,  CP225A: 

Hlchts . 

'.18 

.  10 

.  17 

407 

37.41 

IX  14 

Wrongs . 

.03 

.05 

.02 

407 

28.  55 

8  .  bO 

Scale  Heading,  CP037A: 

High  is . 

*.15 

.38 

.17 

4.11 

44.84 

853 

Wrongs . 

-.05 

-.05 

-.05 

431 

8.05 

A  81 

r pul ::il  UiuLsonlng,  CI211UX2: 

HIgl'itS . 

.08 

.10 

.10 

300 

34).  84 

14.58 

Wrongs .  . 

-.07 

.08 

-.08 

3'A) 

14.  41 

11.20 

.••peed  of  Identification,  Cl’OlOC: 

Hlghts . . . 

*.13 

.25 

.13 

4  48 

73.34 

11.23 

WlOfgS .  . 

.  03 

.01 

.04 

4  46 

3.47 

1.03 

S jkii  i.iv.iiiot),  CT818A: 

1 

Uighu . 

.08 

.33 

.  10 

408 

53  03 

11.87 

\\  rongs . 

.08 

.01 

.08 

408 

J.  87 

A  05 

Visual  Memory,  CI5I4A: 

ItlghU . 

*.13 

.07 

.  14 

4 16 

76.  57 

1154 

W  rungs . 

»  -.  10 

-.  10 

-.  11 

445 

33. 1.2 

14.(0 

Hadnr  Pieferencc  II . 

.00 

.11 

.01 

428 

1.78 

0. 02 

Kiperni  eiitnl  Usycnmnotor  7'ests: 

t  In  ek  List  1  ml  Setting,  MiMel  A  (N’o  Code  No.). 

.03 

.08 

.07 

3.’i? 

77.  fO 

9. 13 

Complex  Cooirllimtlim,  C.M701K . 

.07 

.  12 

.08 

3.28 

5.5.  ID 

7.  17 

Hull*  Control,  CMS.'.U . 

.01 

.07 

.05 

358 

402.  40 

02  01 

Se.f  P  icing  Dlscrinilnntlon  Head  I'll  Time, 

C  1'till K  Modified .  . . 

.07 

.00 

.07 

3 19 

111  08 

44.25 

7  hnistciiie  Two-Hand  I’irrMllt,  CMS10A . 

.01 

.01 

.01 

30i 

C.’7,  57 

70. 33 

Visual  Coincidence  Cl'CI3l)3  Modified . 

-.03 

.07 

-.03 

323 

1 10.  73 

IX  12 

1  I  »s>  leetli.it  variable,  2  =  c<iursc  grade,  and  3™  rudar  stonlne. 

1  Cxinriiil  (or  tange  n  ,->i  ciimi  imsed  on  radar  Manlne.  Sic  footnote  7  on  pise  230  for  for rau Hi  used. 
71  <•  uitriMilet.iI  Mauipml  .}■••.  n.tli.ri  of  i  dar  .Muiuno  was  I.Tti  based  on  0,513  humbardlers  who  look  in# 
rmhr  ot<o-i\cr  M-ii-eiioii  h.ittoiy  prior  to  1  April  1845. 

1  l  !>i*  i?  i  .iii.i  mnl  stand. .rd  ilex  intioii.i  of  me  slmilniM  a;i<l  air-crew  classification  battery  tests  obtained 
fo  nj  •nn.pns  nf  nnei.pf.fnd  aviation  -.tudi-tits  lire  pri-inted  in  table  11.5.  7  ln.io  tuny  In*  compared  villi 
t;.«*  i:it  iii>  mid  .'ImnPiiil  ili  vmtlni.i  of  tne  pre'elil  samples  to  uni irati-  the  union:,  t  of  *a-Ii  dlon  that  Ola-ruled 
on  the  Midei.ts  1'iinr  to  entering  radar  uln.TYi-r  training.  Tin*  ui.ijority  of  the  students  In  lh.j  present 
*  u:  pa  s  in  in  ti  ,ti  .1  nt  tin-  Me.  l;rni  mil  Psyihnlinrkal  l-.x  mu  l:i  tn>;  Units  ro  the  Mint  Mm  .it.onM  be  lulupared 
"  iln  tl.i  •<•  cit.inii.cii  fiom  Mi'ilu’i.l  m .ol  Pay  etiological  Unit  administrations  of  ihu  betlcry. 

1  Mrri.fli m;t  at  the  l-p,-m-nl  level. 

ant  r.:  the  5-|>i-iit-iil  lei  el. 

'lie  v.  i.i'U  o  -e  of  the  validity  cwRlrlenls  was  tested  la-fore  lliey  were  rotln  led  lo  two  figures.  This 
O'  hie  lent  uni)  later  ones  upp.ar  lo  he  sign  meant  after  roun.li.it;  hut  were  Hot  sUullkunt  when  tinted  at  1  tllee 
U'V'hul  places. 


Similar  data  for  tho  navigator  sample  nro  presented  in  tablo  11.9 
and  the  statistics  from  tho  validation  against  average  circular  err*  a* 
are  given  in  tablo  11.10.  Signs  of  all  correlations  involving  circular 
error  have  been  changed  so  that  a  positive  cocllieient  indicates  associa¬ 
tion  of  goodness  of  circular  error  with  goodness  of  te.->t  score.  Tho 
significance  of  these  data,  ns  well  as  of  the  multiple  regression  data  to 
follow,  will  bo  discussed  in  tho  last  section  of  this  chapter.  There, 
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Vnrluhlo 


Sltuilnca: 

Jlnmlmrillor  stniilnr . .  ... 

Nut li'.-.lor  slniilnc .  . . 

Pilot  htillllne 

Hiiil.'ir  Dl.'irvif  Selni  Inn  natt-ry: 

t '<.nr>ltl.iv:r  it.  .clilu:.  CI’22IA  nr  H.  . 

O  rilln-eiipe  1  III'  1 1 in  till  I'  ill,  (.‘I'M 7 A . 

I'n! i ••  r  1 1  Orlfiiiiiiiun.  IT'WA . 

si  iiiii  Oru'iiiuiidti  i.  cr.'/Oin  . 

1  lit<  I  nr  Preference  i . 

Hieler  Miniliir 

Aircrew  t 'lie  wlleatlnii  Huttery  (printed  losts): 

Illmnupliieiil  Dalit  It  lank,  Navigator  Score, 

CKUI2I>  . 

Hingi.iplilral  I), till  lllnnk,  Pilot  Score,  C’K.ffl2  .  . 
iJint  innl  I  utile  Iti  ielliu:,  CPG22A  anil  Cl’021  A  . . . 

(iineml  Information.  CK&OSK . 

Instrument  t’uliiprcheii'liiii  I,  CIG1SH . 

I ti> t miitvit  1  Comprehension  II,  CI61GH . 

M ntlir unities  A,  CI7H2K . 

Mutlii  injties  H,  CI'2I)0C  . 

Mo  liuniru!  Principles,  CP0O3I1 . 

Heading  t 'iiiiipri'licit-siim,  ( ’  I  <>  II 1 1 . 

Spatial  Orieiilulion  I.  CP. Villi . 

Spatial  Orientntlon  II,  CPIVI3H  . 

Aircrew  I’liit  lillcutlnii  Hnttery  fpsyehomotor  lcsls): 

Complex  Coordination,  (  Midi  A . 

Dim  rimlnallon  Knirtlon  Time,  C.'I’OII  L) . 

Kinder  Dexterity,  CM  I  IflA . 


Hnlnry  Pur.-nlt.  C'l’linll 
It i|i !■  li  r  C'nt'trol,  CM  120H 


Two-Hand  (.'iKirillnutliiii,  CM  1 01 A . 

Kxjirrlmenin’  I'rlnteil  Tests: 

A i  rial  <>i  li  ntutlon,  C P.VJOA: 

m  ins .  . 

Wrongs . 

Aren  VI  imllintlun,  CP91.4A: 

I  tilth  til . 

W  rones . 

Air  Cun  s  Vocnbulary  (1912): 

Klglits  ...  . 

Wrongs . 

Comnu  s  Orientntlon,  ClOGOA: 

llieh  in . 

Wrongs  . 

Ksiliimtloii  of  Pength,  (’PAHA: 

Klght.s . 

W  mints  . 

Flight  <tri  ntutlon,  CP.'2-iA: 

ltle Ills  . 

Wrongs  . . 

Meeli. villi  ill  Iiiforiinitinn,  ClOOSl): 

Hi  Ills  . 

W  rones  .  . . .  • 

Meui"i>  for  Inweliiiaiks,  CI.M0AX2: 

Iti'lltS . 

Wrmi  s  .  .  . 

Ninn-  rn  il  Dpi  rations,  CI7H2HX1: 

Kleip.s  . 

\\  run  -s  .  . 

Putt,  i  n  foiopri  In  ie  loti,  Cl'Ml'IA: 

Kl.'Ilts  . 

Wrongs 

l’att-in  I  n  1 1  Hint  Ion ,  CP920A: 

HIT.ts  . 

Wrmi 's  .  . . 

I’olur  i  ;rl  1  Coordinate,  OPSI9I1: 

It  I, tilts  ..  . 

Wroties  ..  . 

PoMlli.ii  Dili  ntntlon,  Cl'tVJiiA: 

HI  lits  . 

\\  rones  ...  . 

Hnl!  i  K-tlin.itlon,  Cl' 22.' A: 

I’.lelits  . 

Wmn-s  . 

Scale  I  Ie. i  ling,  CP637A: 

It  wins  . 

\\  rn"  its  . 

Spatial  lo-wonlm:,  (’121111X2: 

Klelits  . . 

\\  rongs  . 

See  fiwiliiule-  ill  etui  of  tnble. 
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220 
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220 

.os 

.  14 

.IH 

220 

-.01 

-.02 

-.02 

2*Ji* 

* .  10 
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220 
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220 

.00 
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.20 
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226 

.11 

.  13 
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220 

-.04 
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-.01 

220 

.  12 

.02 

.  13 

•220 

-.01 

.02 

.00 

220 

.01 

.01 

-.01 
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-.01 

-.04 
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T.\  m .  k  11  —  l'rod  iict-innnmit  corn  lotions  hi  limn  (hr  nut  nr  observer  course 

ijnitlc  ii fit  si’cclion  variables  for  the  nnviyator  snmpte  in  Validation  Study 
'll  -Continued 


Viu  table 

r,i 1  | 

r„> 

.'ll  1 

.V 

Mr  an* 

SO  1 

rlnn  nt.il  Ptinb  <1  Tests  Continued 
si,.  -l  Hi  Ideiitillfuiion,  ('I'tliot.': 

Bights . 

Wronps . . 

-0  10 
-.01 

0. 

.07 

-0.01 
—  .02 

14  ft 

1  0) 

:s  w 
\  .**7 

h.  13 

1  io 

h'jKji  LcKMlion,  t'I’bIBA: 

Uip'it.i . 

.01 

.44 

.11  i 

m 

m 

9  *7 

.  <12 

.03 

.03 

14*5 

10.  II 

7.  .to 

Vi-tul  Memory,  CI5MA: 

Itiglin  . 

40 

l  .  .  .1  .  | 

w 

i  i  1 

i  |  —  ..Meet  Ion  variable,  2“ grade.  :i n <  1  3-r.tdar  staninc. 

i  Curr-elel  f  >r  range  restriction  !>.»*»•  1  on  r:el  lr  st  mine.  Fee  fisotnute  7  on  page  218  for  formula*  usol. 
Unri  -lri'  le  1  si  in. I  ir  I  deviation  of  radar  si. mine  was  1.77  Ii.lvI  on  3,7u3  navigators  who  took  the  radar 
observer  selection  battery  prior  to  I  April  IVli. 

■  See1  f.xdiiule  3  to  table  1 1.8, 

<  it  ti  111  rout  at  the  If-iH-rcent  level. 

•  Fieinllrunt  ut  the  l-i'ervent  level. 


Taiilk  11.10. — Product-moment  correlations  between  aerrayr  circular  error  and 
selection  variables  for  the  miviyator  sample  in  Validation  Study  II 


Variable 

rn1 

M  1 

.Cl  1 

1 

X 

Mean  * 

SD> 

Ftanlncs: 

l'.oiiiburdlcr  stonine . 

«  0.  18 

0.  14 

0.  10 

228 

7.30 

1  43 

Nm  igator  stinluo.  . 

*.  23 

.18 

.24 

220 

7.  M 

l.as 

Pilot  st/udne  . 

.  10 

.  12 

.11 

220 

a  33 

1.72 

Hnil.ir  Olisi  rver  Selection  Ihittery: 

Coordinate  IteilillMt,  CP22IA  or  14  . 

.00 

.TH 

.02 

220 

13.1. 14 

10  27 

l)-i illoseope  Interpretation,  CPM7A . 

.tvs 

.32 

.flj 

220 

31.18 

r,  irj 

J'litlt  tti  Ori  'hint lull,  t  ‘  i'MtiA  . 

.<U 

.  :e.i 

.01 

220 

3  V  10 

0  V  j 

Spatial  Orientation  1,  t’l'AOIIi . 

-.04 

.  1.1 

-.02 

220 

40.  s  2 

i.  to 

Itailnr  I'refi  rencc  I . 

.  11 

-.24 

.  10 

220 

K.M 

.41 

.02 

.01 

220 

8. 00 

1  00 

Alrtn-w  ('las  ll'.entltm  Iluttery  (prlnled  tests): 

Illovrapldml  Data  Blank,  Navigator  Score, 

('  Kuil’l )  . 

.  10 

.08 

.  10 

228 

23.  78 

3  VI 

Illorrupl.leal  Data  Blank,  Pilot  Stxire,  CK002D... 

.00 

.01 

.  10 

220 

27.  07 

8,01 

Dial  and  T tilde  Heading,  CIY.22A  and  <.'1*621  A... 

*.  IH 

.  :u> 

.17 

220 

4 1. 1: 

7.  IJ 

th  ii.  ral  1  nlortmtUun,  CKA0AK . 

.  10 

—  .09 

.  10 

41.0  V 

I.V  12 

Instrument  Comprehension  1,  CItllSH . 

.00 

-.  u 

.to 

220 

K.  US 

2.  V3  | 

In-trune  nt  Cotnprehetisloii  11,  ClfdOIl . 

.  12 

.  14 

.  12 

r.u 

30.  V7 

II.  20 

Mathematics  A,  CI7tr.’K . 

«.  10 

-.02 

.10 

220 

IV  10 

V  04 

M utl.r lU'i tics  II,  (' I'.WiC . . 

«.  II 

.1)4 

.14 

■ry, 

21.  18 

10.  At 

.M  rehat, l,-.d  Principles,  C  P'Xnll  . 

.  11 

.  II 

220 

33  13 

9.  At  | 

Ite  el ii.e  Comprehension,  CP.14II  . 

.11 

.07 

.  II 

220 

2.1.  TO 

It  83 

Fpatl  il  Orientation  1,  CP. Ml II . 

.09 

.  33 

.09 

220 

X\  M 

4  73 

Kp  t!  .1  Orl.  nt  itlon  1 1,  I’P.’aiII  . 

-.01 

.20 

.10 

220 

20.  13 

8.00 

Alrtr.w  ( *l:e sifietition  Battery  (psycl.oniotor  tests): 

Ciiujpl.-v  Coordi'i atinll,  ( '  M  7111  A  . 

.ai 

.  13 

.ai 

220 

;a  i  s 

iaii 

.  01 

.  10 

.02 

‘Sj'i  1 

M  h7 

5.  rJ  i 

H’  nr  It.  Merity,  C  M  1  ir.  A . . . 

.01 

.  02 

.01 

2V>1 

:,4  :*i 

in  i>v 

Hoi  ary  Pin?  uit.  C  1*41011  . 

.0.8 

AC 

.  US 

;v*» 

At  .1 

V  KJ 

Uitihli  r  Contr.il,  CMP.Mll . . 

.01 

.00 

.01 

'/:•* 

U\  U) 

V  73  1 

Tv.  o  l  1  and  Ci.'ordiit-tli>>n,  CM  101 A . 

.11 

,U1 

.11 

220 

AI.  17 

11.  S.1 

ip<  ri?r.r:it  .1  Printed  Tests: 

1 

Ai  r  at  Orientation,  C  PA20A: 

lilvhts  . 

.  12 

.  17 

.  13 

17V 

21  13 

3  t'J 

Wrongs  . 

-.04 

-.01 

-.01 

I7V 

3  4V 

Area  VPuallrr.tlon,  CPMJA: 

j 

ItiLViU  . 

34  1 

34 

Alt  Curt'S  Vocabulary  (1042): 

1 

. 

0 

0 

Comf.  i's  Orientation,  CP'/OA: 

KlrhM  . 

0 

0 

K«liiti.,iiun  of  length,  CPM1A: 

It li-hts  . . 

-.04 

.3d 

-.01 

17V 

w 

k  71 

W rones  . 

.  12 

.  !U 

.  12 

17V 

27.  Ai 

V  37 

Flight  Orientation,  CP.'.lMA: 

ItChts  . 

.02 

-.  18 

.01 

,15 

40  71 

7.  37 

Wrongs . 

.  28 

.Vi 

.20 

38 

j  ft.  fO 

4.3  ; 

See  footnote?,  at  cml  of  table. 
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Tahiti;  11.10.  -Product-moment  corrdntic ;i.<  hrtic  <’n  nrernye  circular  error  end 
Hcliction  variables  [or  the  navigator  .mm /tie  in  Validation  Study  II — <’on 


Variable 

r,i  1 

ru  1 

r^ll  1 

iV 

•Mr  mi  1 

SD  i 

Experimental  Printed  Tests  — Continued 

Mecbanleal  Information,  C1!a)6H: 

UlghM  . 

0  10 

0.01 

0.  II 

2iM 

10.0.5 

ft.  h7 

W  rone)  .  . . 

10 

-  HI 

-  10 

'S2* 

1 1 . 93 

Memory  for  Hnudmark),  C1510AX2: 

ltU'b i.)  . 

0 

Wrong)  .  . 

0 

Nunn  rli  il  0  iteration),  C*  170211 UC 1 : 

ltl  :bt)  . 

-bt 

.  10 

-.04 

140 

41  ,55 

8  78 

Wroni'i  .  .  . . 

AA 

.  13 

.(« 

116 

5.  31 

2.  U 

r.tlli  rn  ( 'omi, reborn, Ion,  CH803A: 

Right.)  . 

-.09 

.10 

-.03 

178 

21. 44 

4  M 

Wiong)  . 

.  U3 

.02 

.  03 

178 

8.20 

&.  M 

Pattern  Identification,  CP820A: 

Right)  . . 

.01 

.20 

.01 

44 

40.  41 

7.73 

-  02 

-.21 

— .  U2 

44 

7.  18 

0  14 

Polar  (IrM  Coordinate,  Cl’8190: 

HI)  lit) . 

.09 

u 

.09 

115 

02.  30 

12. 10 

Wroni')  . 

-.11 

.05 

-.  II 

115 

3126 

7.01 

ii  Orientation  CP620A: 

jhLl . 

.04 

.  25 

.  05 

179 

219.  10 

39. 42 

Wrong.)  . 

-.01 

.  00 

-.  01 

178 

11.05 

13.25 

Ratio  animation,  CP225A: 

HI, :lil.) . . 

.  13 

.31 

.  13 

150 

43.05 

11.84 

Wrung) . 

.02 

.Ot 

.02 

13) 

20.74 

8.73 

Scale  Heading,  CP637A: 

1!  bt) . 

.05 

.44 

.00 

no 

51.70 

9.02 

Wrung) . 

.05 

.04 

.05 

146 

8.04 

4.76 

Spatial  Reasoning,  C121IDX2: 

Right) . . 

.00 

.  11 

.06 

75 

37.03 

1.5  04 

Wrong) . . 

-.09 

-.oo 

-.09 

70 

17.03 

1079 

Speed  of  Identmcallon,  CI’OIOC: 

Right) . 

-.10 

.28 

-.  14 

140 

75.09 

8. 13 

Wrong) . 

.02 

.07 

.02 

140 

1.57 

1.10 

Spot  location,  C1’8I8A: 

Hlg'it) . 

.02 

.44 

.03 

140 

00.  25 

9.57 

Wrong) . 

-.03 

.03 

-.03 

146 

10-  41 

7.30 

Visual  Memory,  CIS14A: 

HIghU  . . . 

40 

40  1 

. 1 

. 

i  l-relrcllon  variable),  2-avernge  circular  error,  and  3-radar  slonlne. 

»  I’urri'ctt'il  for  more  restriction  havd  on  radar  sinning.  Sen  footnote  7  on  page  236  for  formula)  used. 
Uncorrect e<l  vl  deviations  of  radar  slanine  wus  1.77  based  on  3,703  navigator)  who  took  the  radar 

observer  reliction  nailery  prior  to  1  April  1014. 

■  ;;.'e  footnote  3  to  Tnblo  11.8. 

•  Slgnlllcant  at  llin  5-|>erccnt  lovol 

•  Significant  ut  the  1- percent  lovol. 


evidence  will  bo  summarized  in  terms  of  which  abilities  arc  indicated 
to  be  important  to  success  in  radar  observer  training. 

A  comparison  of  tables  11.8  and  11.9  shows  that  more  statistically 
significant  correlations  were  found  for  the  bombardier  samplo  than  for 
tho  navigators.  Nine  coefficients  are  significant  at  the  1-percent  level 
for  bombardiers  and  an  additional  11  are  significant  ut  tho  5-pcrcent 
level.  Only  ono  of  tho  navigator  coefficients  reached  tho  1-pcrcent 
level  and  an  additional  six  reached  the  5-percent,  level.  In  interpret¬ 
ing  these  tables,  several  differences  between  tho  two  samples  must  bo 
remembered.  Tho  bombardier  sample  is  relatively  largo  (N  =  G29) 
while  tho  navigator  samplo  is  much  smaller  (N  =  22G).  Also,  tho 
criterion  for  tho  bombardiers  is  probably  more  reliable  than  that  for 
the  navigators  since  tho  former  is  based  to  a  greater  extent  upon  scores 
from  standardized  tests  and  performance  checks.  Tho  less  reliable 
criterion  would  cause  an  attenuation  of  tho  validity  coefficients  for  the 
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,i  iN  jgatur  sample.  It  is  further  10  In*  not i  <1  that  tin*  navigators  are 
j,i  obsbly  more  highly  selected  than  bomba nl icrs  on  the  basis  of  psycho- 
],, .dial  funct  ions  that  are  important  to  since.-s  in  radar  observer  t  rain¬ 
ing.  The  restriction  of  range  on  valid  functions  would  attenuate 
validity  eoeflieients  for  tests  measuring  those  functions.  There  is  sumo 
i-viflenee  for  this  in  the  job  analysis  of  the  radar  observer  which  indi¬ 
cates  that  the  fund  ions  involved  are  more  closely  related  to  navigation 
than  to  bombing.  Fm  (her  evidence  comes  from  the  stat  istically 
significant  difference  in  average  course  grade  reported  in  Validation 
Study  I,  which  is  in  favor  of  the  navigators. 

Multiple  Regression  Statistics 

Selection  uf  v<iri<il>hs. — Two  multiple  regression  equations  were 
computed,  one  for  the  bombardiers  and  another  for  the  navigators,  both 
w  ith  the  course  grade  criterion.1®  Variables  were  selected  for  inclusion 
in  these  regression  equations  primarily  on  the  basis  of  having  statisti¬ 
cally  significant  correlation  with  the  criterion.  The  bombardier  and 
navigator  stanincs  were  included  for  both  samples  although  the  navi¬ 
gator  stnnine  did  not  have  significant  validity  for  the  navigator  sample 
against  the  course  grade  criterion  in  the  second  study.  The  navigator 
stnnine  did  correlate  significantly  with  average  circular  error  and,  in 
the  first  validation  study,  with  course  gnule. 

Testy  were  included  in  the  equations  if  their  uneorrected  correlation 
with  the  course  grade  criterion  was  significant  at  the  5-percent  or  1-per- 
ceiit  level  and  was  based  upon  more  than  115  cases.  Experimental 
tcMs,  for  which  both  total  right  and  total  wrong  scores  were  validated, 
were  included  if  either  the  rights  or  wrongs  correlated  significantly 
with  the  criterion.  The  scores  obtained  from  the  administration  of 
Spatial  Orientation  I,  CP501B,  in  the  radar  observer  selection  battery 
were  included  in  both  equations  because  they  almost  reached  the  5- 
pereent  level  of  significance,  in  both  validation  studies  and  for  all 
samples.  Compass  Orientation,  Cl 000 A,  and  Pattern  Identification, 
CPSgOA,  were  not  included  in  the  computations  for  the  bombardier 
sample  although  they  bad  significant  validities.  Their  inclusion 
would  have  greatly  increased  the  number  of  International  Bu.-iness 
Machine  tabulator  runs.  They  were  eliminated  as  the  tests  nearest  tho 
f>  percent  level  of  significance.  Three  tests  without  significant  valid¬ 
ity,  Coordinate  Heading,  CIMTlA  or  B,  Numerical  Operations, 
C1702I3X1,  and  Seale  Beading,  CPG57A,  were  included  for  the  navi¬ 
gator  sample  partly  because  they  were  the  only  tests  who  e  validities 
became  relatively  large  when  corrected  for  range  restriction  based  o  i 
radar  stanine  and  partly  because  it  was  impossible  to  include  them 
v-'itli  little  increase  in  calculating  time. 


w  Thcn'i  ntutljllco  iverc  computed  by  S/Sgt.  Johimton,  Cl").  Kfllcy,  eno  Cap!.  WlltUra  Y. 
Loqu. 
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Tr entmcut  of  rights  and  wrongs  score':. — It  is  well  known  that  a  Ic.-q 
.score.  com  post'd  of  weighted  rights  :nul  wrongs  .scores  often  has  higher 
validity  limn  the,  rights  score,  alone.  To  investigate  this  inedibility 
correlations  between  rights  and  wrongs  scores  were  obtained  for  all 
experimental  tests  used  in  the  bomharuier  regression  equation.  Zero- 
order  multiple  correlations  were  computed  for  each  test  yielding  the 
validity  that  would  he  obtained  by  giving  rights  and  wrongs  scores 
optimal  weights.  The. data  used  in  these  computations  and  tho  result¬ 
ing  nndtiple  correlations  are  presented  in  table  11.11.  It  will  be  noted 
that,  with  the  exception  of  Visual  Memory,  the  validity  coefficients  for 
the  best  combinations  of  rights  and  wrongs  scores  are  ecpial  to  or  only 
very  little  higher  than  the  validities  of  the  rights  scores  alone.  In  view 
of  these,  results,  it  was  decided  to  use  simply  the  number  of  right 
responses  as  the  score  on  eacli  experimental  test. 


Taiii.k.  11.11. — Multiple  correlations  bcticccn  radar  observer  course  grade  and. 
the  best  ircighteil  combinations  oj  rights  and  wrongs  scores  from  selected  testa 
used  for  bombardier  sample  in  Validation  Study  II 


Variable 

Cnc»rrcctcd  cocdktcnts 

Corrected  cwiaclcnts  * 

rn* 

ru' 

ri» 

Kin 

rii 

r,i 

T\l 

/fi.n 

Aerial  Orientation,  C  1*520 A . 

-0.  89 

-0.09 

0. 12 

0. 13 

-0.  88 

-0.09 

0. 13 

0.14 

Numerical  Operation*,  (’170211X1 . 

.04 

-.03 

.  10 

.  17 

.03 

-.04 

.18 

.18 

1‘nt  a  <  Ida  Oooraimitc,  ('1‘ttluH . 

.03 

.02 

.  10 

.  10 

.  11 

.03 

.12 

.12 

It  ,1  in  IWtltiul Inn,  0  1*225 A . 

-.08 

.02 

.  16 

.  10 

-.07 

.02 

.17 

.17 

?<•»!(•  Hnvlliti:.  (’IVv17A  .  .  . 

-.13 

-.05 

.  15 

.  IS 

-.14 

-.05 

.17 

.17 

Spc,  a  of  Mr  lit  ilk- .it  Inn,  C  1’OtOC . 

.03 

.03 

.12 

.  12 

.04 

.04 

.13 

.14 

Vi  mill  Memory,  ('1514  A . 

.03 

.  13 

.  17 

.02 

-.11 

.14 

.18 

1  Corrected  fur  range  restriction  basc<l  on  radar  stanine.  See  foolnolo  (2)  to  table  U.&. 
*  1  — course  uncle,  2— rights  score,  and  3  -wrongs  score. 


Multiple  regression  statistics  for  bombardier  sample. — The  uncor¬ 
rected  intercorrelntions  and  validities  for  all  variables  used  in  the 
bombard  it  r  multiple  regression  equations  arc  presented  in  table  11.12. 
Tho  corresponding  coefficients,  corrected  for  rungo  restriction  based 
on  radar  stanine,  arc  presented  in  table  11.13.  The  betn  weights  for 
three  regression  equations  were  computed  for  the  bombardier  sample, 
one  including  the  bombardier  and  navigator  .stnnines,  a  second  Includ¬ 
ing  15  tests,  the  selection  of  which  is  described  above,  and  a  third 
including  both  tests  and  stnnines.  The  beta  weights  and  multiple 
correlation  cocflicients  resulting  from  each  of  these  combinations  ait) 
presented  in  tabic  11.11.  The  equations  were  computed  both  from 
uncorrccled  cocflicients  and  from  cocflicients  corrected  for  range 
restriction  based  on  radar  stanine.  As  pointed  out  in  a  footnote  to 
this  table  and  to  tables  11.17  and  11.19  to  follow,  the  multiple  coeffi¬ 
cients  have  been  corrected  for  expected  shrinkage.  However,  the 
correction  made  is  an  underestimate,  since  the  nvnilnblo  formula  does 
not  cover  tho  case  in  which  tho  multiple  is  based  upon  a  smaller  num¬ 
ber  of  more  valid  variables  selected  from  a  larger  total  group.  The 
corrections  arc  presented  in  the  absence  of  a  more  accurate  estimate. 
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Tahij:  11.11. — Multiple  regression  statistics  for  bombardiers,  Validation  ntudy  u 
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Multiple  regression  statistics  for  navigator  sample. — Tho  uncor* 
reeled  intcrcomdulions  nnd  validities  for  all  variables  used  in  tho 
navigator  regression  equations  are  presented  in  tablo  11.15.  Tho 
corresponding  coefficients  corrected  for  rango  restriction,  based  on 
radar  staniue,  are  presented  in  table  11.10.  The  beta  weights  for 
three  regression  equations  were  computed  for  the  navigator  sample, 
one.  including  the  bombardier  and  navigator  stanincs,  a  second  includ¬ 
ing  nine  tests,  and  a  third  including  both  the  tests  and  stanincs. 


Taih.f:  1  l.lH.  —  Unrorrceted  intrrcorrrlations  awl  validities  of  variables  used  in 
computing  multiple  regression  statistics  for  navigator  sample,  Validation 
Study  II 
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'rune.  11. Ill  Cnrri'i'tnl  inh  1 i  nn  rliitimix  nn<t  i  Oiihtii.i  of  rm'i'iWi.t  i  Mi-.f  i>i  com- 
jiuliiuj  multiple  reijn  xxion  /■tatiitiri  /or  iun  i<j<itur  x<nn file,  Viihtlution  Study  11  * 
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Those  weights  and  the  multiple  correlation  coellioicnts  reuniting  from 
each  combination  nro  listed  in  table  11.17.  These  data  and  the  beta 
weights  computed  for  the  bombardier  sainplo  will  be  discussed  in  the 
lust  section  of  this  chapter  in  a  consideration  of  the  abilities  necessary 
to  successful  completion  of  the  radar  observer’s  course. 

Tam.k  11.17. — Multiple  rcffrcsxlon  atatixticx  for  nut  hjators,  Validation  h>l\.d\j  1! 
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1' cal  nation  of  job  analogy  and  facto c  a  pproacha  to  ach  >  tio:\.  ■ 
kiltie  data  from  a  single  investigation  can  provide  no  ih  iuMlo  com 
parboil  of  the  job  analogy  and  factor  approaches  to  :  election  c  t 
search,  it  appeared  of  intercut  to  test  the  relative  elh.eieney  of  tV  ' 

1  fhoih;  in  this  study  of  radar  observer  selection.  Two  suM . 

;  'i  of  imiltiplo  regret  -ion  statistics  were  competed.  One  <o  »l.t 
included  tlio  valid  te.it ;j  among  (ho.-ii  consider  d  to  be  factor  t*- 
hs'ed  on  pages  200  to  207  and  in  footnote  tl  on  pagu  ‘JOG.  Th  u!' 
s*  t  included  tho  valid  tests  among  those  con.-iuei  c(l  to  be  job  an  -h 


tests,  listed  on  pages  207  to  208  and  in  footnote  13  on  page  207.  The 
comparison  was  made  only  for  the  bombardier  sample  since  too  few 
cases  were  available  for  seven  of  the  experimental  factor  to-,ts  to  pro¬ 
vide  for  their  validation  in  the  navigation  sample.  Omission  of  that 
number  of  tests  would  have  put  the  factor  approach  at  a  decided 
disadvantage. 

The  intercorrelations  and  validities  of  the  two  groups  of  tests  are 
given  in  table  11.18.  The  derived  multiple  regression  statistics  are 
given  in  table  11.19. 


Taiii.k  11.18. — / ntcrcorr<  la  lions  and  validities  of  job  analogy  and  factor  ir*i 
batteries,  bombardier  sample,  Validation  Study  II 


Variable 


1 


2.  Di  li  ami  Table  Reading,  CIT.2IA. 
2.  Niinii'i  l>-:il  Operations,  (’170211X1.. 

4.  Polar  drill  Coordinate,  CPH19B . . 

5.  Hole  Reading,  0 1  (137 A . 

0.  Spatial  Oi  lew,  it  ion,  1  (’)  Cl'SOlB. 


Uncorrectcd  eoelllelents 

1 

2 

3 

4 

5 

0 

C.  Cj 

or. 

'  ' 

0.  If, 

0. 19 

0.29 

0.  14 

-0.  (1.3 

S 

O 

0.  1(1;....!  .22  .  IS 
.  10.  .  32  ....  i  .21 


.20!  .  IS 
.Mi  .as 
-.021  .05 


.21'.... 

.  42 j  .2-1 

.01  .02, 


.as1 
.  12 
.34 

.02 


Corrected  eoelllelents  > 


.05  .10  0.20 
.  Oil  .  10;  .  22 
.02'  .  Id!  .47 
.021  .  15l  .22 
. ...  .07!  .201 


V  ° 

Si 

is* 


0.20,0.  33,0.  47,0.  33'0.  20  0.  II 


.27 

.27' 

.35 

.101 


.3.' 

.  48 

.41 

.’is 


.12 

.18 

.13 

.17 

.09 


1 

2 

3 

< 

su 

(P 

1 

2 

3 

4 

5s  * 

U  ^ 

3  F 
o  h 
U 

1.  Aerial  Orientation,  CPA20A . 

2.  N  inner  leal  Operations,  ('1 702BXI . 

-0.05 

-0. 03 

—0.04 

.  25 

0.  Ofl! 

.  17 

0.  12 
.  10 

-o.oi 

-0.01 

o.oo 

.31 

0.  07 
.  18 

0.11 

.18 

3.  Sliced  of  Ideritllleatlon,  CPGIUC _ 

-.04 

.25 

.08 

.  12 

.00 

.31 

.09 

.11 

4.  V'isnal  Memory  CI5I4A . 

.00 

.17 

.08 

.13 

■°7, 

.18 

.09 

.14 

1  Corrected  for  rango  restriction  based  on  Radar  Observer  Selection  Battery.  Sco  footnoto  (2)  to  tabls 

11.8. 

*  Based  on  scores  from  administration  of  Spailul  Orientation  I  In  tbo  Rndnr  Observer  Selection  Battery. 


Taiii.k  11.19. — Multi •  'c  repression  statistics  for  job  antilogy  and  factor  lest 
batteries,  bombardier  sample,  Validation  Study  II 
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•  lbrcd  on  scores  from  administration  of  Spatial  Orientation  1  In  tbo  Radar  Observer  Selection  Battery. 

•  sec  footnoto  4  Tublo  11.14. 
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It  will  be  noted  that  the  four  factor  tests  yielded  a  slightly  higher 
multiple  correlation  than  did  t ho  six  job  analogy  tests.  More  im¬ 
portant,  however,  is  the  fact  that  neither  multiple  correlation  ap¬ 
proaches  the  value  produced  by  all  15  tests  used  for  the  bombardiers, 
shown  in  table  11.14.  Also,  the  beta  weights  in  table  11.14  of  two  of 
the  job  analogy  tests.  Coordinate  Rending,  (T±24  A  or  B,  and  Spatial 
Orientation  I,  CP501B,  are  large  even  though  the  valid  factor  testa 
arc  included  in  the  same  regression  equation.  This  latter  fact  suggests 
that  the  validated  factor  tests  do  not  adequately  sample  the  factor 
content  of  the  criterion. 

The  evidence  from  this  comparison  of  the  factor  and  job  analogy 
approaches  to  selection  rest  research  suggests  that  neither  should  bo 
depended  upon  exclusively.  The  conclusion  indicated  is  that,  for  the 
present,  joint  use  of  both  approaches  .should  he  made.  Until  addi¬ 
tional  factors  have  been  isolated  and  a  greater  number  of  factor 
measures  developed,  it  will  be  wise  to  supplement  the  latter  with  job 
analogy  tests. 

INDICATED  VALIDITY  OF  FACTORS 

The  validation  statistics  and  multiple  regression  data  presented  in 
the  foregoing  sections  provide  a  basis  for  some  tentative  conclusions 
regarding  the  factors  involved  in  the  task  of  the  radar  observer,  'l  est 
validity  is  a  result  of  factorial  content  common  both  to  the  test  and  tho 
criterion  task.  Given  tho  test  validities  and  factor  loadings,  it  is 
possible  to  make  preliminary  estimates  as  to  tho  relative  importance 
for  success  in  radar  observer  training  of  the  factors  considered  in  this 
chapter  and  chapter  4.  Such  conclusions  will  suggest  which  selection 
tests  measure  important  or  valid  factors  and  therefore  merit  further 
research. 

This  section  summarizes  the  data  pertaining  to  validity  for  each 
factor.  It  should  he  remembered  that  tho  conclusions  readum  a  re 
limited  by  the  tests  validated  and  that,  consequently,  the  list  of  (..<  (m  s 
is  necessarily  incomplete.  Other  qualifying  limitations  of  tho  data 
should  also  be  kept  in  mind.  Tho  most,  important  of  these  i-.  v’no 
undetermined  reliability  of  tho  criteria  used  in  the  validation  slt.du  s. 
Since  the  degree  to  which  tho  validity  coefficients  are  attenuate  l  by 
the  unreliability  of  the  criteria  is  unknown,  the  conclusions  to  bo  draw  n 
from  them  are  necessarily  doubtful.  A  second  limitation  is  due  to 
the  fact  that  the  validity  data  are  obtained  from  a  sample  of  students 
who  have  already  undergone  selection  resulting  in  what  is,  ,■  Argo 
extent,  an  unknown  amount  of  restriction  of  range  in  ability.  As 
will  be  pointed  out,  it  is  probable  that  the  apparent  lack  of  validity 
of  some  factors  may  be  a  reflection  of  this  restriction.  A  third  .'imi¬ 
tation  is  tiie  factorial  complexity  of  certain  of  the  valid  teds;  for  such 
tests  the  factor  content  contributing  to  tho  test  validity  is  dtfkenh  to 
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infer.  A  fourth  limitation  lies  in  the  fact  that  the  factor  content  of 
some  of  the  validated  tests  is  neither  accurately  nor  completely  de¬ 
scribed.  It  is  impossible  to  conclude  definitely  from  tie  validity  or 
nonvalidity  of  such  tests  whether  or  not  the  lectors  they  aro  thought 
to  measure  aro  valid.  Finally,  the  validation  sample  is  small  and 
sampling  errors  will  necessarily  influence  some  of  the  conclusions 
reached.  Nevertheless,  the  analysis  has  been  made  in  this  form  be¬ 
cause  of  its  interest  as  a  general  approach  to  the  task  of  summarizing 
the  implications  of  a  group  of  validity  coefficients. 

The  factor  summaries  aro  presented  in  alphabetical  order.  Defini¬ 
tions  of  the  factors  discussed  will  bo  found  in  chapter  4.  In  each 
summary,  a  table  is  presented  which  lists  significant  factor  content 
and  validity  coefficients  for  nil  validated  tests  which  have  loadings 
in  the  factor  larger  than  the  arbitrarily  selected  lower  limit  of  C  1. 
The  factor  loadings  given  in  the  tables  arc,  for  the  most  part,  approxi¬ 
mations  based  upon  factor  analyses  of  the  July  1943  and  November 
1913  air-crow  classification  batteries.  Pertinent  ii  formation  from  the 
multiple  regression  equations  will  be  presented  in  the  text.  The  beta 
weights  referred  to  will  be  found  in  table  11.14  and  11.17.  For  reasons 
already  given,  validity  coefficients  against  course  grade  in  Validation 
Study  II  will  receive  somewhat  greater  emphasis  in  the  analysis  than 
thoso  in  Validation  Study  I. 

Evidence  for  Factor  Validity 

Length  estimation. — Estimation  of  Length,  CPG13A,  is  thought  to 
measure  the  length  estimation  factor.  Its  loadings  aro  unknown 
because  it  has  never  been  included  in  a  factor  analysis.  However,  since 
it  showed  no  validity  in  the  samples  to  which  it  was  administered,  it 
is  likely  that  the  length  estimation  factor  has  no  validity  for  radar 
observer  success. 

Mechanical  experience. — Table  11.20  presents  the  factor  loadings 
and  validities  of  tests  measuring  the  mechanical  experience  factor. 
AVhilo  the  evidence  is  to  some  extent  contradictory,  it,  in  general, 
favors  the  conclusion  that  this  factor  has  some  validity  for  radar 
observer  training.  Mechanical  Information,  CI905B,  when  included 
in  the  navigator  regression  equation,  makes  a  unique  contribution  as 
shown  by  its  sizable  beta  weight  (table  11.17).  This  can  bo  explained 
only  by  its  mechanical  experience  factor  content.  This  same  test, 
on  the  other  hand,  shows  no  validity  in  the  bombardier  sample  of 
Validation  Study  II.  The  validity  of  General  Information,  as  shown 
under  the  discussion  of  pilot  interest  below,  is  probably  best  explained 
by  its  mechanical  experience  content.  The  validity  of  Mechanical 
Principles  and  Biographical  Data  Blank,  pilot  score,  in  only  one  of 
the  5  samples  in  which  they  were  validated  makes  doubtful  any  con¬ 
clusions  based  upon  them. 


Table  11.20.— Summary  of  evidence  pn  Mining  to  ralidity  of  the  Mechanical 
Experience  factor  (or  radar  observer  training 
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Memory  /. — Table  11.21  presents  the  factor  loadings  and  validi¬ 
ties  of  the  test  thought  to  measure  the  memory  I  or  rote  memory 
factor  No  significant  validity  was  obtained  for  the  test.  Memory 
for  Landmarks,  CI310AX2.  Since,  in  Validation  Study  Ia.  none  of 
the  navigators  had  scores,  the  test  was  validated  only  for  the  bom¬ 
bardiers.  This  meager  evidence  indicates  a  lack  of  validity  of  mem¬ 
ory  I  for  radar  observer  training. 


Table  11.21. — Summary  of  evidence  pertaining  to  validity  of  the  Memory  l 
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Memory  //. — Table  11.22  presents  the  factor  loadings  ami  validi¬ 
ties  for  the  test  thought  to  measure  memory  II  or  vi.-mul  memory. 
This  test,  Visual  Memory,  CI314A,  was  validated  in  Validation  Study 
II  for  tho  bombardier  sample  only,  because  too  few  navigators  had 
scores  to  warrant  computing  a  validity  coefficient.  For  the  bombnr- 
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Taiiijc  11.22. — Rumman/  of  evidence  pertaining  to  validity  of  the  Memory  // 

factor  for  radar  observer  training 
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dicr  sample,  the  rights  score  of  Visual  Memory  had  a  validity  coeffi¬ 
cient  of  0.13,  significant  at  the  1  percent  level,  while  the  wrongs  score 
had  a  coefficient  of  —0.10,  significant  at  the  5  percent  level.  This  evi¬ 
dence,  although  based  only  on  one  sample,  suggests  strongly  that 
memory  II  has  slight  but  stable  validity  for  radar  observer  training. 

Numerical  Facility. — Table  11.23  presents  the  factor  loadings  and 
validities  of  the  tests  measuring  the  numerical-facility  factor. 
The  validity  of  Numerical  Operations,  C1702BX1,  a  relatively  pure 
measure,  is  good  evidence  for  the  validity  of  the  numerical-facility 
factor.  That  the  test  did  not  have  significant  validity  for  the  naviga¬ 
tor  sample,  Validition  Study  II,  is  probably  explained  by  the  great 
selection  in  numerical  ability  known  to  have  operated  on  the  navi¬ 
gators.  In  all  regression  equations  in  which  it  is  included  (tables 


Taiii-E  11.23. — Summary  of  evidence  pertaining  to  validity  of  the  Numerical 
Facility  factor  for  radar  observer  training 
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31,11  and  11.17),  Numerical  Operations  maintains  r  sizable  beta 
avoi^zlit  which  can  only  bo  explained  by  the  validity  of  numerical 
facility.  . 

It  is' not  clear  what  is  indicated  by  the  validity  of  the  fuctorially  com¬ 
plex  tests  which  have  loadings  in  numerical  facility.  Mathematics  A, 
Cl 702 1*\  besides  its  numerical  facility  loading,  has  loadings  in  verbal 
comprehension  and  in  a  factor  tentatively  named  “mathematics  back¬ 
ground.”  The  validity  of  verbal  comprehension  is  doubtful,  ns  will 
be  shown  below.  This,  plus  the  fact  that  no  measure  of  “mathematics 
background”  is  in  the  bombardier  regression  equation  (table  11.14), 
makes  it  appear  that  the  negative  beta  weight  of  Mathematics  A  in  tiiat 
(•(illation  is  best  explained  in  numerical  facility  content  held  in  com¬ 
mon  with  Numerical  Operations.  This  is  based  on  the  assumption 
that  a  valid  test  will  not  contribute  to  a  multiple  correlation  if  purer 
measures  of  its  valid  factors  are  also  in  the  regression  equation.  The 
validity  of  Mathematics  B,  CT200C,  on  the  other  hand,  seems  to  come, 
in  part,  from  the  Reasoning  I  content  since  it  has  a  sizable  beta  weight 
(tables  11.14  and  11.17)  oven  when  included  with  Numerical  Opera¬ 
tions.  Other  evidence  for  the  validity  of  Reasoning  I  is  given  on  page 
•2(1 1.  The  valid  factor  content  of  Dial  and  Table  Reading,  ClMiiiilA 
and  CR021A,  may  be  inferred  from  i' shot  a  weights  in  the  bombardier 
and  navigator  regnvsion  equations.  In  the  former,  it  lias  a  zero  beta 
weight  when  included  with  relatively  pure  measures  of  numerical 
facility  (Numerical  Operations),  Space  I  (Aerial  Orientation),  and 
perceptual  speed  (Speed  of  Identification).  In  the  navigator  equa¬ 
tions,  however,  perceptual  speed  is  represented  by  Spatial  Orientation 
I  and  numerical  facility  is  represented  by  Numerical  Operations,  but 
no  measure  of  Space  I  is  present.  In  this  case,  Dial  mid  Table  Read¬ 
ing  has  a  siziblo  beta  weight  which  seems  to  indicate  that  its  Spare  l 
content  yields  the  bulk  of  its  validity.  The  data  seem  to  warrant  the 
conclusion  that  the  numerical  facility  factor  has  validity  for  radar 
observer  training. 

Perceptual  speed. — Table  11.24  presents  the  factor  loadings  ami 
validities  of  tests  measuring  the  perceptual  speed  factor.  The  valid¬ 
ity  <»f  Speed  of  Identification,  CBOIUO,  for  the  bombardier  sample, 
Validation  Study  IT,  supports  the  validity  of  tbo  perceptual  .speed  fac¬ 
tor  since  this  factor  accounts  for  a  substantial  portion  of  the  tv  t’s 
variance.  In  the  bombardier  regression  equation  (table  11.14),  Speed 
of  Identification  has  a  large  beta  weight.  On  the  other  hand,  no 
Hiitable  explanation  can  ho  given  for  its  lack  of  validity  in  th  •  n...  iga- 
tor  sample  of  Validation  Study  II.  Spatial  Orientation  I  (semes  from 
administration  in  the  radar  observer  selection  battery)  was  included 
in  both  regression  equations  even  though  it  did  not  have  nueomvled 
validity  coefficients  significant  at  the  5-porceut  level.  In  both  cases, 
>t  obtained  sizable  beta  weights.  This  was  somewhat  surprising  in 
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Taiile  11.21.— Summary  of  evidence  pertain  ini)  to  validity  of  the  Perceptual 
Speed  factor  for  radar  observer  training 
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tho  caso  of  the  bombardier  equation,  since  significant  factor  content 
other  than  perceptual  speed  is  not  known  for  cither  Speed  of  Identifi 
cation  or  Spatial  Orientation  I.  While  the  available  evidence  is  not 
uncontradictory,  it  seems  in  general  to  favor  the  tentative  conclusion 
that  the  perceptual  speed  factor  is  valid. 

Pilot  Interest. — Table  11.25  presents  the  factor  loadings  and  valid¬ 
ities  of  the  single  test  measuring  the  pilot-interest  factor.  The  fac¬ 
torial  complexity  of  this  test,  General  Information,  CE505E,  pre¬ 
cludes  drawing  any  clear-cut  conclusion  as  to  its  factor  validity.  It 
yielded  a  relatively  large  beta  weight  in  the  bombardier  regression 
equations  (table  11.1-1)  even  though  better  measures  of  space  I  nnd 
perceptual-spei  d  each  had  sizeable  weights  in  the  same  equation.  This 
may  be  taken  to  mean  that  tho  validity  of  General  Information  is  due 
to  factor  content  other  than  space  I  or  perceptual  speed.  If  tho 
validity  of  General  Information  were  completely  due  to  its  space  I  or 


Taiii.e  11.25. — Summary  of  evidence  pertaining  to  validity  of  the  Pilot  Interest 

factor  for  radar  observer  training 
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perceptual-speed  content,  it  could  not  be  expected  to  add  to  the  pre¬ 
dictive  value  of  nil  equation  which  includes  better  measures  of  tlieso 
factors.  Its  total  valid  variance  in  this  case  would  be  explained  by 
content  held  in  common  with  the  other  measures  and  it  would  add 
nothing  to  their  combined  predictive  power.  This  view,  plus  the  lack 
of  supporting  evidence  for  the  validity  of  verbal  comprehension  con¬ 
tent,  leads  to  the  conclusion  that  the  validity  of  General  Information 
is  due  either  to  its  mechanical  experience  or  its  pilot-interest  loadings. 
On  the  basis  of  job-analysis  information,  mechanical  cxi>ericnco  seems 
to  be  n  more  likely  explanation  for  validity  than  does  pilot  interest. 
However,  until  these  deductions  are  supported  by  additional  empirical 

evidence  the  validity  of  the  pilot-interest  fnctor  must  remain  in 
question. 

Psychomotor  Coordination. — Table  11.20  presents  the  factor  load¬ 
ings  and  validities  of  tests  measuring  the  psychomotor  coordination 
factor.  Neither  Rotary  Pin-suit,  CIM  10B,  nor  Complex  Coordination, 
CM701A,  produced  a  significant  validity  coefficient-  This  apparent 
lack  of  validity  indicates  that  individual  differences  in  psychomotor 
cc ordination  are  not  important  to  the  radar  observer's  task  in  training. 


Table  11.20. — Summary  of  evidence  pertaining  to  valid  it  if  of  the  Psychomotor 
Coordination  factor  for  radar  observer  training 
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Psychomotor  Precision. — Table  11.27  presents  the  factor  loadings 
and  validities  of  tests  measuring  the  psychomotor  precision  faetjr. 
Finger  Dexterity,  CM  1  IGA,  yielded  validity  significant  at  the  6  percent, 
level,  for  the  navigator  sample,  validation  study  I,  but  for  none  of 
the  remaining  four  samples  in  which  it  was  validated.  Discrimina¬ 
tion  Reaction  Time,  CPGllC,  is  also  valid  in  only  one  of  the  samples. 
Its  validity  in  this  instance  is  difficult  to  explain  because  of  its  facfi.;  1 
complexity.  In  the  bombardier  regression  equation  (table  11.1-1)  t  o 
test  lias  a  sizable  beta  weight  even  though  measures  of  space  I  (Aeri.d 
Orientation) ,  numerical  facility  (Numerical  Operations),  and  percep¬ 
tual  speed  (Speed  of  Identification)  are  also  in  the  equation.  Ihiu 
seems  to  indicate  the  validity  of  its  p.sychomotor  precision  loading. 
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Taiii.k  11.27.— Summary  of  evidence  pertaining  to  validity  of  the  I’m/choinotor 
Precision  factor  for  radar  observer  training 
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However,  in  view  of  the  small  proportion  of  tlie  possible  samples  in 
which  either  of  tho  tests  considered  was  valid,  no  definite  conclusion 
about  the  validity  of  this  factor  could  be  reached. 

Reasoning  /. — Table  11.28  presents  the  factor  loadings  and  va¬ 
lidities  of  tests  measuring  the  reasoning  I  factor.  Tho  validity  of 
Spatial  Reasoning,  CI211BX2,  in  the  navigator  sample,  validation 
study  II,  indicates  some  validity  for  reasoning  I.  This  test,  on  tho 
other  hand,  showed  no  validity  in  the  bombardier  sample  of  valida¬ 
tion  study  II.  When  Mathematics  B,  the  other  test  with  reasoning  I 
loading  is  included  in  regression  equations  with  Numerical  Opera¬ 
tions,  both  receive  sizable  beta  weights  (tables  11.14  and  11.17).  Thi9 
seems  to  indicate  that  the  validity  of  Mathematics  B  is  accounted  for 
either  by  its  reasoning  I  or  its  verbal  comprehension  loadings  in  addi¬ 
tion  to  its  numerical  facility  loading.  As  pointed  out  later,  there  is 
almost  no  evidence  for  tho  validity  of  the  verbal  comprehension  fac¬ 
tor;  hence  the  validity  of  Mathematics  B,  over  and  above  that  duo  to 


Taiii.u  11.28. — Summary  of  evidence  pertaining  to  validity  of  the  Reasoning  I 

factor  for  radar  observer  training 
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numerical  facility  scorns  most  likely  to  be  duo  to  its  reasoning  I  load¬ 
ing.  It  is  possible  that  the  validity  of  another  test,  Heading  Compre¬ 
hension,  0101411,  is  also  due  primarily  to  its  reasoning  I  loading, 
'phis  test,  which  is  omitted  from  table  11. gS  because  its  reasoning  I 
loading  is  only  0.25,  lias  a  verbal  comprehension  loading  of  about  0,05 
and  a  mechanical  experience  loading  of  0.25.  When  included  in  a 
regression  equation  that  also  includes  Mathematics  B  (table  11.14), 
it  has  a  beta  weight  of  zero.  Since  these  two  tests  have  in  common 
only  the  verbal  comprehension  and  reasoning  I  factors  and  verbal 
comprehension  is  assumed  to  he  invalid,  the  conclusion  indicated  is 
that  the  validity  of  Heading  Comprehension  is  duo  to  reasoning  I. 
In  summary,  the  preponderance  of  evidence  indicates  that  reasoning  I 
is  valid  for  radar-observer  training. 

Space  I. — Table  11.20  presents  the  factor  loadings  and  validities 
of  tests  measuring  the  space  I  or  spatial  relations  factor.  The  clearest 
evidence  for  the  validity  of  the  space  I  factor  is  the  validity  of  Aerial 
Orientation,  CP520A,  which  is  thought  to  have  a  major  loading  only 
in  space  I.  Supporting  evidence  is  yielded  by  the  validity  of  Dial  and 
Table  Heading,  CPG22A  and  CP621A,  which  has  been  explained  on 
page  220  primarily  by  its  space  I  loading.  Greatest  doubt  comes  from 
the  lack  of  validity  of  Instrument  Comprehension  II,  CIGIOB,  Com¬ 
plex  Coordination,  CPT01A,  and  Flight  Orientation,  CP52SA.  The 
lack  of  validity  for  Instrument  Comprehension  II  is  particularly 


T.vmx  11.20. — Summary  of  evidence  pertaining  to  validity  of  the  Space  I  (tfuitlal 
relation i)  factor  for  radar  observer  training 
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troublesome  since  it  has  major  loadings  in  two  other  factors  for  but], 
of  which  there  is  good  evidence  of  validity.  The  data  from  Discrim- 
ination  Reaction  Time,  CPGllD,  and  Two-IIaiul  Coordination  ! 
CM101A,  are  ambiguous  because  these  tests  are  complex  factorial^ 
The  validity  of  Discrimination  Reaction  Time  has  already  been  in¬ 
terpreted  as  probably  due  primarily  to  psyehomotor  precision.  Tli? 
ambiguity  of  this  evidence  prohibits  nny  conclusions  at  this  time 
regarding  the  validity  of  space  X.  It  is  concluded  that  space  I  may 
have  low,  positive  validity  for  predicting  success  in  radar  observer 
training. 

Space  II. — Table  11.30  presents  the  factor  loadings  and  validi¬ 
ties  of  the  test  thought  to  measure  the  Space  II  or  rotational  space 
factor.  This  test,  Position  Orientation,  CP'52GA,  yielded  no  sig. 
nificant  validity  for  either  sample  of  Validation  Study  II.  A  lack 
of  validity  of  Space  II  for  radar  observer  training  is  indicated. 


Tabu*  11.80. — Summary  of  evidence  pertaining  to  validity  of  the  Spare  II  factor 

for  radar  observer  training 
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Verbal  comprehension. — Table  11.31  presents  the  factor  leadings 
and  validities  of  tests  measuring  the  verbal  comprehension  factor. 
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The  fairly  pure  measure,  Air  Corps  Vocabulary  ( 10-12),  failed  to 
produce  significant  validity.  However,  this  test  was  only  validated 
for  the  bombardier  sample,  Validation  Study  II,  since  none  of  tho 
navigators  used  in  the  study  had  Air  Corps  Vocabulary  scores.  Tho 
validity  of  Reading  Comprehension,  the  only  other  adequate  mt'.iMiro 
of  verbal  comprehension,  is  probably  explained  by  factor  content 
other  than  verbal  comprehension,  specifically,  by  Reasoning  I.  This 
is  deduced  from  the  fact  that  its  beta  weight  is  near  zero  in  u  bom¬ 
bardier  regression  equation  (table  ll.l  t)  in  which  it  is  the  only  test 
with  a  significant  verbal  comprehension  loading.  All  evidence  taken 
together  seems  to  indicate  that  verbal  comprehension  has  no  validity 
for  radar  observer  training. 

Visualization. — -I  able  1 1  ..‘52  presents  the  factor  loadings  and  va¬ 
lidities  of  the  tests  measuring  the  visualization  factor.  The  validity 
of  0.18,  significant  at  the  1-percent  level,  produced  by  Mechanical 
Principles,  CI903B,  in  tho  navigator  sample,  Validation  Study  I,  is 
tho  only  validity  yielded  by  a  test  having  a  major  loading  in  tho 
visualization  factor.  Since  this  validity  may  be  explained  by  the  high 
mechanical  experience  loading  of  Mechanical  Principles,  it  appears 
likely  that  visualization  has  no  validity  for  radar  observer  training. 


T.ujie  11.32. — Summary  of  evidence  pertaining  to  validity  of  the  Visualization 
factor  for  radar  observer  training 
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Summary  of  Validity  of  Factor x. — On  tho  basis  of  the  preceding 
summaries  of  tho  data  for  each  factor,  the  factors  may  be  grouped 
under  several  headings  on  the  basis  of  estimated  validity  for  rudm* 
observer  t mining.  Five  of  tho  fnctom  considered  have  been  judi  d  to 
bo  valid;  these  ate  mechanical  experience,  memory  II  (visual  memory) , 
numerical  facility,  perceptual  speed,  and  reasoning  I  (general  reason* 
ing).  The  evidence  in  tho  case  of  threo  of  these  factors,  memory  II, 
numerical  facility,  and  reasoning  I,  is  relatively  clear-cut  but  varies  k» 
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amount.  The  evidence  for  the  validity  of  the  remaining  two  factor 
mechanical  experience  and  perceptual  speed,  while  not  altogether  un- 
contradictory,  is  still  preponderantly  favorable.  The  weight  of  avail¬ 
able  evidence  points  to  a  lack  of  validity  for  the  following  six  factors; 
Length  estimation,  memory  (rote  memory),  psychoniotor  coordina¬ 
tion,  space  II  (rotational  space),  verbal  comprehension,  and  visuali- 
zation.  For  three  factors,  pilot  interest,  psychoniotor  precision,  ami 
space  I  (spatial  relations),  the  evidence  from  u'fTercnt  sources  is  too 
ambiguous  to  make  any  conclusion  possible. 

SUMMARY 

The  results  of  two  validation  studies  are  presented  in  this  chapter. 
In  the  first  study,  the  air-crew  specialty  stanine  scores,  the  tests  in  the 
air-crew  classification  battery,  and  the  tests  in  the  radar  observer  se¬ 
lection  battery  were  validated  against  course  grades  assigned  by  the 
training  authorities  at  Boca  Raton,  Langley  Field,  and  Victorville. 
The  course  grades  were  based  partly  upon  informal  tests  and  instructor 
ratings  and  partly  upon  standardized  tests  and  performance  checks. 
The  validation  samples  consist  of  205  bombardiers  and  381  navigators. 

For  the  bombardier  sample  in  the  first  study,  six  variables  had 
positive  validity  coefficients,  significant  at  the  5-perccnt  level  or  1-per¬ 
cent  level;  the  navigator  and  pilot  stanincs;  navigator  score  of  Bio¬ 
graphical  Data  Blank,  CEG02D;  Dial  and  Table  Reading,  CPG2*2A 
and  CPG21A;  General  Information,  CE505E;  and  Mathematics  A, 
CI702F.  For  the  navigator  sample,  nine  variables  had  positive  valid¬ 
ity  coefficients,  significant  at  the  5-percent  or  1-pcrcent  level ;  the  bom¬ 
bardier  and  navigator  stanincs;  pilot  score  of  Biographical  Data 
Blank,  CEG02D;  Dial  and  Table  Reading,  CPG22A  and  CPG21A; 
Mathematics  B,  CI20GC;  Mechanical  Principles,  CI903B;  Reading 
Comprehension,  CIG01II;  Spatial  Orientation  II,  CP503B;  and 
Finger  Dexterity,  CM11GA. 

The  second  study  included  the  variables  in  the  two  batteries  used  in 
Validation  Study  I  and  also  an  experimental  selection  battery.  The 
primary  criteria  consisted  of  radar  observer  course  grades  obtained  for 
classes  later  than  those  used  in  Validation  Study  I,  but  before  VJ-duy. 
Course  grades  for  the  G29  bombardier  students  from  Boca  Raton  and 
Langley  Field  were  computed  by  the  Radar  Project  wholly  from 
standardized  test  and  performance  check  scores.  The  course  grades 
for  the  22G  Victorville  students,  all  of  whom  were  navigators,  were 
similar  to  those  used  in  the  fi rst  study,  being  based  primarily  upon 
subjective  ratings  and  informal  quizzes.  The  selection  variables  wort 
also  validated  against  average  circular  error  scores  for  the  Victorville 
sample. 
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For  the  bombardier  sample,  10  variable*  bad  pn-itive  correlations 
with  course  grades,  significant  at  t  lit*  5-pcrecut  or  1-perccnt  level:  the 
bombardier  and  navigator  stanines,  Aerial  Orientation,  (T.VJOA; 
Coordinate  Reading,  CP221A  or  B;  Dial  and  Table  Reading.  C PGA2 A 
and  CPG-lA;  Discrimination  Reaction  Time,  CRG11D:  General  In¬ 
formation,  CE.MVE;  Matin:,  lies  A,  (TTugF;  Mathematics  B, 
CTJdt.C';  Numerical  Operation-.  ClTDJBXl:  l’olar  Grid  Coordinate, 
CPS19B ;  Ratio  Estimation,  C  P'_,'_,5  A ;  Reading  Comprehension, 
CI011II;  Scale  Reading.  CPG37A;  Speed  of  Identification,  CPClbC; 
and  Visual  Memory,  CP5I1A.  These  variables  along  with  Spatial 
Orientation  I,  CP501B,  produced  an  uncorrected  multiple  correlation 
of  0.51  with  the  course  grade  criterion. 

For  the  navigator  sample,  six  variables  had  positive  correlations 
with  coui.-e  grades,  significant  at  the  5-pcreeut  or  1-percent  level;  the 
bombardier  stnnine;  Dial  and  'Fable  Reading,  CI’GJJA  and  (.  lVr.HA; 
Mathematic.s  B,  CT-’OGC;  Mechanical  Information,  CT!H)5B;  Pattern 
Orientation,  CPS1GA;  and  Ratio  Estimation,  CP225  A.  These  vari¬ 
ables  together  with  the  navigator  stanine,  Coordinate  Reading, 
CPTMA  or  B,  Numerical  Operations,  CI702BX1,  Scale  Reading, 
CPC37A,  and  Spatial  Orientation  I,  CP501B,  yielded  an  nucorrected 
multiple  correlation  of  0,36. 

For  the  navigator  sample,  Five  variables  had  negative  correlations 
with  average  circular  error  scores,  significant  at  the  5-pcrcent  or  1-per¬ 
cent,  level:  the  bombardier  and  navigator  stanines;  Dial  and  Table 
Reading,  CPG22A  and  CPG21A;  Mathematics  A,  CI702F;  nnd 
Mathematics  B,  CP20GC. 

The  samples  of  bombardiers  and  navigators  for  which  the  validity 
coefficients  were  computed  find  been  subjected  to  selection  at  four 
points  in  their  training:  before  attaining  aviation  student  status,  at 
classification  center,  at  bombing  or  navigation  school,  ami  prior  to 
entering  radar  observer  training.  Although  it  would  have  been  de¬ 
sirable  to  correct  the  coefficients  for  more  of  the  restrictions,  it  was 
possible  to  correct  them  only  for  the  last  which  was  based  on  the  radar 
stanine.  The  corrected  coefficients  are  estimates  of  the  validities  tho 
tests  would  have  for  the  population  of  graduating  bombardiers  and 
navigators.  In  most  cases  the  corrections  made  little  dilTcmicc. 
Both  corrected  and  uncorrected  coefficients  were  used  to  compute  tho 
multiple  regression  statistics.  The  corrected  coefficients  yielded 
slightly  higher  multiple  correlations  in  most  cases. 

Validity  and  multiple  regression  data  were  summarized  in  terms  of 
the  indicated  importance  of  different  factors  for  radar  oh server  train¬ 
ing,  Five  factors  were  judged  to  be  valid  for  training  success: 
ntcchnnical  experience,  memory  II  (visual  memory),  nmuericnl  fa¬ 
cility,  perceptual  speed,  and  reasoning  I  (general  reasoning). 
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CHAPTER  TWELVE _ _ 

An  Evaluation,  With  Suggestions 
for  Future  Research' 


In  this  chapter,  psychological  research  carried  out  in  rudar-oliscrvcr 
training  will  he  reviewed  both  from  the  point  of  view  of  its  interest 
to  psychologists  and  its  potential  value  to  the  radar-observer  pro¬ 
gram.  Before  undertaking  such  a  review,  it  will  be  helpful  to  recall 
that  the  oriental  ion  of  the  Radar  Project,  like  that  of  similar  projects, 
was  primarily  to  “war  psychology”  and  that  its  first  concern,  con¬ 
sequently,  was  to  produce  for  wartime  use'.  Within  this  limitation, 
however,  the  greatest  possible  effort  was  made  to  insure  the  collection 
of  data  which  would  make  possible  evaluation  of  the  reliability  and 
validity  of  its  instruments. 

The  presentation  within  the  chapter  is  organized  in  the  following 
terms:  (1)  The  development  and  validation  of  selection  tests,  (2)  the 
development  of  proficiency  tests  and  performance  checks,  ('.))  the- 
development  of  methods  for  instructor  evaluation  and  selection,  and 
(4)  research  on  trainers  and  training  methods. 

JOB  ANALYSIS  AND  SELECTION-TEST  RESEARCH 

Chapter  1  of  the  report  presents  a  discussion  of  u  job  description 
and  job  analysis  of  the  task  of  the  radar  observer  in  training.  In  chap¬ 
ter  11  the  results  of  tho  validation  of  selection  tests  are  analyzed;  in 
part,  this  analysis  is  summarized  in  terms  of  the  predicted  validities  of 
psychological  abilities  resulting  from  the  job  analysis.  To  personnel 
psychologists  concerned  with  the  process  of  studying  a  job  from  tho 
point  of  view  of  anticipating  which  te.^ts  will  predict  success,  the  ap¬ 
proach  in  chapter  4  will  be  of  considerable  interest.  In  describing 
the  radar  observer’s  task,  a  distinction  is  made  between  “job  descrip¬ 
tion,”  in  which  the  description  is  made  in  terms  specific  to  a  single  job, 
and  “job  analysis,”  where  description  is  nmde  in  terms  of  general  abili¬ 
ties.  It  is  believed  that  a  reading  of  chapter  4  will  present  convincing 
evidence  of  the  value  of  distinguishing  between  the  t  wo.  As  is  pointed 
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out  (here,  a  job  <lr  miption  is  more  useful  in  the  preparation  of  pro¬ 
ficiency  measures  and  in  planning  t  raining  lv.-eanh  and  research  into 
techniques  and  equipment.  Potli  the  ji.b  d<-eripl ion  and  job  analysis 
arc  Useful  as  a  basis  for  select  ion  test  research,  although  it  is  predicted 
that  sneli  re-ranh  will  more  and  more  come  to  be  based  upon  the  latter. 
Opinion  on  t b is  point  will  be  a  function  of  one's  judgment  as  to  the 
eventual  value  of  factor  theory  for  prediction  of  jon  surer.  s.  Factor 
theorists  hold  that  performance  on  most  of  the  tasks  in  contemporary 
technology  ran  be  explained  by  a  limited  number  uf  independent 
function:-,  or  factors  taken  in  various  combinations  and  amounts.  Tf 
this  proves  to  be  l  rue,  occupational  analysis  and  select  ion-test  research 
in  (be  future  should  consist  of  description  of  jobs  in  terms  of  statisti¬ 
cally  isolated  abilities,  followed  by  validation  of  tests  known  to 
measure  these  abilities. 

The  dillieully  commonly  experienced  by  factor  analysts  in  preparing 
an  adequate  verbal  description  of  a  newly  isolated  factor  gives  rise 
to  a  parallel  diflienlty  for  the  job  analyst.  Verbalizing  one’s  intro¬ 
spections  following  experience  with  the  job  or  observation  of  persons 
undertaking  the  job,  in  itself  presents  difficulties,  but  these  are  greatly 
increased  by  the  problem  of  converting  these  introspections  into 
vaguely  described  psychological  functions. 

In  chapter  1 1  tho  results  of  two  validation  studies  arc  presented. 
In  (he  first  study,  the  air-crew  specialty  stauinc  scores,  tho  tests  in  the 
air-crew  classification  battery,  and  the  tests  in  a  selection  battery  used 
to  screen  radar  observer  students,  were  validated  against  course  grades 
assigned  by  the  training  authorities  at  three  radar  observer  training 
stations.  These  grades  were  based  partly  upon  informal  tests  and 
instructor  ratings  and  partly  upon  standardized  tests  and  performance 
checks,  'flic  validation  samples  consisted  of  205  bombardiers  and  381 
navigators.  The  second  study  included  the  variables  in  the  two  test 
batteries  used  in  Validation  Study  I,  plus  tests  from  an  experimental 
selection  battery.  Tho  validation  criterion  in  this  study  consisted  of 
radar  observer  course  grades  for  classes  subsequent  to  those  used  in 
Validation  Study  I,  plus  tests  from  an  experimental  selection  battery. 
Thu  validation  criterion  in  this  study  consisted  of  radar  observer 
course  grades  for  classes  subsequent  to  those  used  in  Validation  Study 
1,  hut  before  VJ-day.  Course  grades  for  ono  part  of  the  sample,  G29 
bombardier  students,  were  computed  by  the  Radar  Project  wholly 
from  standardized  tests  and  performance  check  scores.  Course  grades 
for  tho  navigator  part  of  tho  sample,  which  consisted  of  226  students, 
were  similar  to  those  used  in  the  first  study,  being  based  primarily 
upon  subjective  ratings  and  informal  quizzes.  For  the  navigator 
students,  tho  selection  variables  were  also  validated  against  average 
circular  error  scores. 


272 


The  most  interesting  multiple  correlation  coefficients  me  believed 
to  be  those  resulting  from  the  second  study,  since,  in  this  stinlv,  n  largo 
number  of  experimental  selection  tests  were  validated  against  a  pro¬ 
ficiency  criterion  composed  primarily  of  standardized  proficiency  tests 
and  performance  checks.  For  the  bombardier  sample  in  this  second 
study,  an  uneorreeted  multiple  validity  coefficient  of  0.51  was  obtained. 
Tbe  parallel  figure  for  the  navigator  sample  was  These  figures 

must  be  qualified  in  several  ways.  In  the  first  place,  the  samples  of 
students  for  which  the  validation  coefficients  were  computed  had  been 
subjected  to  selection  at  four  points  in  their  training  prior  to  adminis¬ 
tration  of  the  selection  tests  in  this  study.  They  had  been  tested  as  a 
prerequisite  to  being  granted  aviation  student  status;  they  wore 
screened  further  at  air-crew  classification  centers;  additional  persons 
were  eliminated  through  training  failure  nt  bombing  or  navigation 
schools ;  mul  still  others  were  eliminated  by  tests  prior  to  their  cntranco 
to  radar  observer  training.  Selection  of  this  sort  undoubtedly  gavo 
rise  to  curtailment  of  abilities,  for  which  it  was  not  possible  to  correct. 
Such  corrected  coefficients  as  arc  presented  consist  only  of  estimates 
of  the  validities  the  tests  would  liavo  had  for  the  total  population  of 
graduating  bombardiers  and  navigators.  It  is  not  possible,  conse¬ 
quently,  to  compare  tost  validities  presented  in  this  report  with  those 
obtained  in  air-crew  classification  centers.  For  tbe  samo  reason  it 
would  bo  possible  from  these  results  to  make  only  approximate  esti¬ 
mates  of  the  validity  of  tests  for  selecting  radar  observers  from  the 
population  of  students  admitted  to  aviation  student  status. 

Tho  second  qualification  has  to  do  with  the  shrinkage  to  ho  ex¬ 
pected  in  tho  multiple  correlations  reported.  While  an  attempt  has 
been  made  to  correct  for  this  shrinkage,  it  is  known  that  the  correction 
affords  an  underestimate  of  the  amount  of  shrinkage  which  will  occur. 
With  this  qualification,  the  shrunken  cofilcients  which  correspond  to 
the  figures  of  0.51  and  0.00  given  ahovo  nrc  0. 17  and  0.21  respectively. 

A  third  qualification  Inis  to  do  with  the  reliability  of  the  validation 
criterion.  According  to  the  best  evidence  available,  this  reliability 
is  probably  not  higher  than  0.10  or  0,50. 

Plans  had  been  made  for  further  experimental  validation  of  selec¬ 
tion  tests.  However,  the.-c  were  interrupted  by  tbe  cessation  of  train¬ 
ing  which  accompanied  the  end  of  the  war.  It  bad  l>con  decided  that 
the  most  fruitful  areas  to  te.-t  beyond  those  already  sampled  would  lm 
those  of  interests,  attitudes,  and  emotions.  Biographical  data  tests 
which  had  proven  to  bo  valunblo  in  other  uir-crcw  special* i» were  lo 
have  been  tried  out.  Future  selection  test  research  with  radar  ob¬ 
servers  should  probably  emphasize  tests  in  these  fields. 

As  indicated  above,  the  validation  statistics  were  analyzed,  in 
part,  from  the  point  of  view  of  hypotheses  advanced  in  connection 
with  tho  job  analysis.  The  tests  validated  were  divided  into  three 
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groups:  ono,  composed  of  t(  Is  nieasiiiit  g  hnown  factors;  a  second 
of  t  !.s  o  pocinlly  con- !  rtic'.cd  lo  simuhue  a  part  of  t’.o  rular  ob¬ 
server’s  job;  and  a  third,  of  to  ts  wlii<  li  (oiihi  In-  oh  arly  filled  into 
neither  of  these  categories.  Of  the  group  of  factor  '.r-tq  four  hlU] 
validity  coellirieiil  ignilicunily  different  from  zero.  For  tile  second 
group  of  te  t  • ,  called  job  analogy  t ■  ~ t ,  there  wore  ■  ix  significant  co¬ 
efficients.  1  lie  fun  t  1  art  or  t  c>ls  yielded  a  mill  1 1  jdc  con  el  at  ion  of  u  •_>  g 
while  the  six  job  analogy  tests  gave  a  multiple  of  O/.’l.  Neither  of 
tin -e  value-;  approach  the  multiple  of  I ).  1 5  obtair.eil  with  tho  total 
gi oii[»  of  te.t.s.  Thu-;,  while  the  approach  tlurngli  available  factor 
tests  win  <  *<  [  u :  1 1 1  y  .- ilia  !•->.- fill  with  that  of  con  .-it  net  ion  ol‘  new  jol> 
analogy  tests,  it  is  clearly  ad\  Fable  to  u  e  both  in  combination.  Two 
inferences  may  lie  drawn.  One  is  that  an  inadopiate.  coverage  of 
factor  le.-ts  was  vailable;  the  second  is  that  conipl  \  selection  tests  of 
the  job  analogy  type  measure  abilities  not  covered  by  available  factor 
term.  In  the  latter  ease,  a  factor  study  of  the  new  job  analogy  It  s 
should  ho  illuminating. 

/V  note,  of  inteivst  in  connection  with  the  use  of  factor  tests  in 
m  lection  test,  re  card  i  has  to  do  with  the  customary  at  tit  udo  toward 
the  expected  size  of  individual  test  validities.  7~.ictor  tests  con- 
sislently  yielded  low  validities,  often  from  0.10  to  0.30.  Tlie.se  values 
alone  would  be  discouraging,  were  it  not  for  the  fact  that  batteries 
of  relatively  uneorrehited  tests  with  such  validities  yield  useful 
multiple  correlations.  For  example,  for  the  bombardier  sample  in 
tl io  second  validation  study,  reported  in  chapter  11,  15  tests  with 
validities  ranging  from  0.07  to  0.1G  yielded  a  multiple  correlation  of 
0.'15.  Similarly,  for  the  navigator  sample,  0  tests  with  validities  rang¬ 
ing  from  0.08  to  0.18  produced  a  multiple  of  0.31. 

Also  nf  interest  to  occupational  analysts  is  the  degree  of  success 
which  lKvnmpa.iiod  attempts  to  predict  the  validity  of  various 
psychological  abilities  on  tho  basis  of  the  job  analysis.  In  chapter 
1 1,  nil  possible  evidence  was  accumulated  which  related  to  the  validity 
of  each  factor.  This  evidence  unfortunately  is  very  slight  in  many 
cases  and  conflicts  in  others.  In  addition,  it  is  based  upon  rather 
small  samples.  "With  these  qualifications,  a  rough  summary  of  success 
in  predicting  factor  validity  may  be  made  ns  follows:  Of  the  five 
factors  for  which  there  seemed  the  most  positive  evidence  of  appre¬ 
ciable  validity,  one  had  been  predicted  to  be  among  the  most  valid, 
three  had  beei  predicted  to  be  among  those  with  relatively  high 
validity,  one  had  been  predicted  to  be  among  those  with  relatively  low' 
validity,  none  had  been  predicted  to  be  among  those  with  lowest 
validity,  nnd  none  bad  been  predicted  to  have  no  validity.  Of  the  six 
factors  for  which  there  seemed  to  be  mo>-t  definite  evidence  for  lack  of 
validity,  three  bad  been  predicted  to  have  no  validity,  one  had  been 
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pi  edict  oil  to  be  among  tlio.se  with  lowest  validity,  one  had  been  pre¬ 
dicted  to  bo  among  those  with  relatively  low  validity,  one  had  been 
predieted  to  be  among  those  with  relatively  high  validity  and  none 
had  been  predicted  to  bo  among  the  most  valid. 

PROFICIENCY  .MEASUREMENT 

The  Radar  Project  was  established  shortly  iftov  the  Army  Air 
Forces  embarked  upon  a  greatly  expanded  program  for  the  train¬ 
ing  of  radar  observers.  This  expansion  occurred  in  the  fall  of  101-1’ 
and  followed  a  year  of  increased  success  in  strategic  bombing  with 
radar  in  tho  European  Theater.  Supervisory  training  personnel 
under  tho  leadership  of  Col.  William  M.  Garland  wero  dissatisfied 
with  the  methods  of  proficiency  measurement  that  had  been  hurriedly 
improvised  and  were  desirous  that  new  types  of  proficiency  measures 
bo  devised.  This  situation  was  of  interest  to  tho  Radar  Project  not 
only  because  of  the  expressed  need  for  ndequnto  methods  of  evaluating 
student  proficiency  but  also  because  it  appeared  there  was  available 
no  adequate  criterion  against  which  to  validate  selection  tests.  Of 
the  possiblo  criteria,  one,  the  pass-fail  criterion,  was  eliminated  beeauso 
the  demand  by  the  operational  nir  forces  for  radar  observers  was  so 
great  as  to  prohibit  the  failure  of  all  save  the  most  inferior  students, 

I  Another,  instructor  grades,  appeared  likely  to  be  of  doubtful  value 
beeauso  rapid  expansion  of  training  necessitated  the  use  of  many  in¬ 
structors  with  no  previous  teaching  experience  and  others  with  little  or 
no  motivation  to  teach.  A  third,  bombing  accuracy,  was  made  im¬ 
practical  by  the  lack  of  sullieient  photographic  equipment  at  tlm  nular 
j  training  stations. 

As  a  result,  a  comprehensive  battery  of  standardized  proficiency 
measures  was  constructed  and  put  into  wide  use.  This  battery,  which 
is  described  in  chapters  5  and  (»,  consisted  of  five  printed  proficiency 
tests  and  six  individually  administered  performance  cheeks.  Four  of 
the  tests  were  used  at  intermediate  points  in  the  course  to  measure  pro¬ 
ficiency  in  specific  phases  of  training,  while  a  fifth  served  ns  a  final 
comprehensnc  examination.  Four  of  the  performance  checks  meas¬ 
ured  proficiency  on  ground  trainers,  while  two  measured  aerial  per¬ 
formance.  Of  the  latter, one  was ndmini.-tered  midway  through  aerial 
training,  while  the  other  served  as  a  final  cheek  on  aerial  performance. 

Practical  circumstances  were  such  that  it  was  possible  to  compute  the 
reliability  only  of  the  imspeedcd  sections  of  printed  proficiency  te.,ts. 
The  reliability  of  the  speeded  tests  and  of  the  performance  cs  ;  is 
not  known.  Fstimatesof  the  reliability  of  tho  course  grade  ba^-d  nj  on 
the  battery  of  standardized  measures  indicate  that  it  lies  between  O.-tO 
ami  0.50.  On  the  basis  of  otlv.  r  evidence,  which  also  must  be  qualified 
considerably,  it  seems  doubtful  that  this  value  is  higher  than  the  relia¬ 
bility  of  the  grades  assigned  before  the  standardized  battery  was  put 
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into  use.  While*  recognizing  llio  n-M-rvat  i ons  with  which  this  compari¬ 
son  must,  be  made,  it  is,  on  the  other  lm  ml,  olcar  that  the  reiiabilit y 
iiidi'  utetl  for  the  standardized  moa  uivs  leaves  nmol;  to  ho  desired. 

Analyses  of  the  interrelations  between  various  proficiency  measures 
are  suggest  ivi  of  probable  differences  in  reliahility  between  the  primed 
tests  and  tin*  performance  checks.  It  was  found  in  these  studies, 
reported  in  chapter  X,  that  t.-sts  coveiin;;  dis.-dmil.ir  material  from 
dillcrent  phases  of  the  course  correlated  more  highly  with  each  other 
than  did  tests  and  performance  checks  covering  similar  subject  mate- 
rial.  While  more  than  one  inference  is  pov-iblc  from  this  finding,  it  b 
at  least  consistent  with  the  hypothesis  that  printed  tests  are  more 
reliable  than  are  performance  checks. 

The  experience  gained  by  the  Ihular  Project  in  attempting  to  solve 
difficult  ies  of  standardized  administ  rat  ion  in  a  large-scale  achievement* 
testing  program  may  be  of  value,  in  other  similar  situations.  Ditlicul- 
t  ies  fell  primarily  into  two  categories.  One  of  these  followed  from  the 
necessity  to  depend  for  test  and  check  administration  upon  operating 
per.onnel  with  no  formal  measurement  training.  The  number  and 
extent  of  the  dilliculties  which  can  hinge  partly  upon  the  motivation 
and  partly  upon  the  comprehension  of  measurement  problems  by  such 
personnel  were  underestimated.  After  many  of  these  dilliculties  be¬ 
came  apparent,  a  system  was  established  whereby  selected  administra¬ 
tors  were  organized  at  each  training  station  into  specialized  examiner 
hoards.  This  was  done,  partly,  in  order  to  limit  the  number  of 
personnel  that  would  have  to  be  trained  in  standardized  administration 
and,  partly,  to  provide  for  more  systematic  supervision  of  examiner 
personnel.  Observation  of  the  operat  ion  of  this  plan  suggested  that  it 
would  produce,  adequate  administration,  providing  that  examiners 
were  selected  care  full)  and  indoctrinated  thoroughly  as  to  the  impor¬ 
tance  of  their  task. 

A  second  category  of  difficulties  had  to  do  with  the  standardization 
of  testing  conditions,  particularly  in  connection  with  the  administra¬ 
tion  of  individual-performance  checks  on  ground  trainers  and  in  the 
air.  Initial  contacts  with  the  situation  and  with  training  pei’sonnel 
led  the  project  to  believe  that  the  more  important  variables  could  lx* 
controlled.  Further  experience  showed,  however,  that  control  of  vari¬ 
ables  such  as  weather,  route,  crew,  radar  equipment,  aircraft,  etc.,  was 
administratively  dillicult  and,  under  wartime-training  conditions, 
probably  impossible.  Adjustment  to  this  realization  took  the  form  of 
“standardized’  treatment  of  instances  in  which  departure  from  stand¬ 
ard  conditions  occurred.  A  procedure  was  developed  whereby  an 
examiner  was  required  not  only  to  check  each  item  as  the  performance 
check  progressed  hut  also  to  keep  a  Jog  of  the  conditions  under  which 
he  found  it  impossible  to  evaluate  an  item  according  to  the  standardized 
specifications.  This  log  served  as  a  basis  for  u  joint  decision  on  the 
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qui-jit  ionahle  item  by  the  examiner  and  the  head  of  the  examiner  Ixmrd. 
The  latter  individual  developed  a  set  of  rules  governing  tvpiral  cxeep- 
t ions  to  routine  administration,  in  terms  of  whirh  lie  conhl  make  what 
appeared  to  be  consistent  and  reliable  decisions,  rnfortiiuatel v,  it  was 
never  possible  to  evaluate  systematically  the  efficacy  of  this  procedure 
from  the  point  of  view  of  impnn  ement  in  performance-check  reliabil¬ 
ity. 

While  some  hope  is  held  for  improvement  of  reliability  through 
adjustments  such  as  those  described  above,  it  now  seems  clear  that 
another  safeguard  should  also  be  attempted.  It  is  well  known  that  the 
reliability  of  a  composite  of  separate  correlated  measures,  each  with 
low  reliability,  can  be  increased  by  increasing  the  number  of  measures 
in  the  composite.  It  is  probable  that  this  could  have  been  accomplished 
in  the  radar  observer  training  program  by  adding  the  battery  of  stand¬ 
ardized  proficiency  measures  to  existing  grades,  rather  than  substitut¬ 
ing  it  for  these  grades. 

An  unexpected  and  beneficial  outcome  of  the  effort  to  standardize 
proficiency  measurement  was  improved  standardization  of  the  curricu¬ 
lum  and  of  instruction.  Supervisory  personnel  faced  with  the  tusk  of 
providing  for  standard  testing  conditions  found  themselves  applying 
pressure  for  increased  attention  to  a  thorough  and  systematic  training 
routine.  A  standardized  measurement  program  cannot  function  in  a 
training  situation  where  the  instructional  material  varies  irregularly 
from  time  to  time  and  front  school  to  school. 

Several  measuring  instruments  were  developed  which  will  be  of  spe¬ 
cial  interest  to  test  technicians.  One  section  of  the  fund  comprehen¬ 
sive  examination  was  built  around  the  concept  of  a  simulated  radar 
observer  bombing  mission.  This  section  consisted  of  interrelated 
items  which  required  the  solution  of  navigational  and  bombing  prob¬ 
lems  at  various  points  along  a  typical  mission  route.  A  full-scale  navi¬ 
gational  chart  and  photographs  of  the  radar  scope  were  used  in  tlm 
student's  computations.  This  section  of  the  test  was  prepuml  in 
n spouse  to  a  strong  conviction  of  training  authorities  that  continous 
navigation  on  a  mi.-sion  requires  something  more  than  can  he  measured 
by  tests  of  the  separate  skills  involved.  They  believe  that  organiza¬ 
tional  abilities  and  possibly  emotional  qualities  are  required  for  the 
integrative  aspects  of  comlueting  the  mission. 

In  addition  to  this  section  of  the  comprehensive  examination,  tliero 
were  three  other  sections,  each  measuring  separate  navigational  skills 
such  ns  use  of  computers,  use  of  charts,  and  interpretations  of  scope 
photographs.  The  c  three  siibtc-ts,  when  combined,  correlated  from 
0..7)  to  0X0  with  the  suhre.-t  containing  the  simulated  mission.  How¬ 
ever,  it  was  not  possible  io  determine  whether  they  accounted  for  all 
of  its  nonem.r  variance,  since  the  reliability  of  the  various  snbte  t.s 
was  not  kiv)'  n.  It  would  have  been  of  practical  intere.-t  to  be  ablo 
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to  answer  this  question,  since  the  simulated  mission  was  very  time 
consuming  per  item  in  comparison  with  the  simpler  sub  tests.  Con¬ 
siderable  timo  could  have,  been  saved  if  the  simulated  mission  were 
to  1  in ve  boon  eliminated  with  assurance  that  the  shorter  sections  ade¬ 
quately  mcasuied  the  total  task.  Since  all  of  the  subtests  were  speeded, 
it  would  hn\e  been  necessary  to  have  prepared  equivalent  forms  in 
order  to  obtain  n  satisfactory  reliability  estimate. 

A  second  point  of  interest  tv)  test  technicians  is  the  type  of  per¬ 
formance  check  developed  for  use.  on  the  supersonic  ground  trainer 
and  in  the  air.  Practical  circumstances  in  these  two  instances  forced 
the  measurement  of  proficiency  while  the  student  carried  out  a  typical 
performance  of  the  total  job  for  which  he  had  been  trained.  Physical 
and  temporal  limitations  of  the  training  situation  made  it  impossible 
to  break  up  this  total  task  into  a  number  of  independent  items.  In 
addition,  the  project  was  faced  with  a  linn  conviction  on  the.  part  of 
training  personnel  that  any  such  division  of  the  total  task  into  inde¬ 
pendent  items  would  remove  from  it  many  of  the  important  skills  for 
which  measurement  was  desired.  Consequently,  during  a  perform¬ 
ance  cheek  given  on  supersonic  trainer  or  in  the  air,  the  radar-observer 
student  simply  Hew  a  typical  mission.  Standardization  was  attempted 
by  having  the  mission  flown  over  one  of  two  matched  standard  routes 
on  the  assumption  that  this  would  present  each  student  with  an 
equivalent  set  of  navigational  and  bombing  problems.  Standardized 
directions  to  tho  student  were  provided.  The  examiner  was  furnished 
standardized  instructions  for  evaluating  performance  in  terms  of  spe¬ 
cific  items,  each  of  which  was  accompanied  by  defined  standards  and 
tolerances.  Tho  items  in  such  checks  consist  of  evaluations  of  per¬ 
formance  at  convenient  and  crucial  points  in  tho  course  of  a  mission. 
Adjustment  to  this  type  of  measurement  situation  led  to  the  develop¬ 
ment  of  si  vend  techniques  which  ure  described  in  chapter  6.  For 
example,  it  was  found  possible  to  evaluate  certain  important  features 
of  performance  only  in  tonus  of  outcomes  observed  later  in  the  missioa. 

Certain  generalizations  about  the  measurement  of  tho  performance 
resulted  from  tho  project’s  intensive  experience  with  proficiency 
measures.  Although  too  lengthy  for  description  here,  these  are  pre¬ 
sented  in  detail  in  chapter  7. 

HADAK  BOMBING  CIRCULAR  ERROR 

It  was  impossible  for  tho  Rndar  Project  to  make  an  analysis  of 
radar  bombing  circular  error  until  near  tho  end  of  the  war.  Very 
little  data  were  available  earlier  because  of  lack  of  photographic  and 
ground  radar  scoring  equipment.  The  most  significant  annlysis  of 
such  data  as  did  become  livnilablo  consisted  of  determining  its  re¬ 
liability.  The  results,  ns  presented  in  chapter  9,  indicate  that  circular 
error  scores  have  low  reliability;  coefficients  range  between  0.20  and 
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ii  ;;n.  Correlation  of  :t \  circular  fi  rm-  \\  i l ] t  course  grades  ;» rt*. 

:i  n*l  <>. •-!<<.  TliC'f  liunro  .-u^ol  lli:i(.  while  a  wrap* 
(.|H  lii’  c  rri  >r  w  « » 1 1 1  <  1  he  tif  lit  1 1  e  value  if  used  alone  a.-  a  criterion  of 
|ipi!i  leiicv,  it  "1  ii »i 1 1*  1  lie  c« »ii" u It* r**« I  for  n~(>  as  part  of  a  composite 
crit-  rioii. 

( '• . i u i >a ;  i:  on  o f  st  ini ie>  1 1 f  ra<lar  lioinliiiio  >i  ore>  with  *»t »i*l it*s  of  vis¬ 
ual  loiuliino  scores  sii^u'e-i  that  tin-  former  are  somewhat  more  re  1  i - 
able.  A  possible  i - .x 1 1 1 ; 1 1 1 ; 1 1  ion  for  this  ditrereiice  is  that  there  is  levs 
\ a r : a  1 » i  1  i t _v  amon<;  students  in  visual-bombin';  proficiency  than  in 
pillar  bombing.  Consequent  ly,  the  ililfereitces  between  stmlents  in 
visual  bombing  are  mole  easily  obscured  by  variations  in  factors 
mu  flat  ed  to  bonibiiio  ability. 

.V  iiimiber  of  siiooest  ions  have  been  made  in  chapter  !>  regardin'; 
the  improvement  of  reliability  of  radar  bombin';  scores.  These  are 
primarily  concerned  with  the  standardization  of  bombing  conditions 
and  with  the  problem  of  obtaining  more  complete  records  of  bombing 
scores  for  each  student.  ’The  scoring  of  bombing;  inns  with  the  aid  of 
"loiiml  radar  i  n-ta  Hat  ions  promises  much  toward  the  elimination  of 
ei lor  in  the  scoring  process. 

Of  interest  to  students  of  measurement  is  a  discussion  in  chapter  0 
of  the  commonly  used  methods  of  determining  circular  error  relia¬ 
bility.  'The  inadequacy  of  methods  which  did  not  take  account  of 
day  to-day  variations  are  pointed  out.  The  conclusion  reached  is 
tli.it  the  only  adequate  method  of  estimatin';  circular  error  reliability 
is  that  of  correlating  scores  from  odd  bombing  missions  with  scores 
from  even  bombin';  missions,  allowin';  bonibiim  conditions  to  varv 
as  i  hey  do  under  routine  t  raining  com!  it  ions. 

INSTKICTOK  SKI.KCTIO.N  AM)  KVA  I  A' ATIO.N 

Although  extensive  plans  were  made  for  research  in  instructor 
'•ele.-tion  and  evaluation,  most  of  the  plans  were  cancelled  by  the 
t'  i  luinat  ion  of  host  i  lit  ies.  Follow i 1 1 jx  a  job  description  and  job  analysis 
of  i  adar-observer  instruction,  an  experimental  selection  battery  was 
cli"  fii  but  never  validated.  Scales  for  the  rutin*;  of  instructors  by 
students  were  eoii-tnu  ted  and  their  reliability  determined.  It  is  u 
fa i '  of  some  simiilicaiice  for  future  re-carch  that  the-e  scales  were 

P 

f"i  nd  to  have  very  hij;h  reln.bilit  y.  A  possible  implication  of  this 
is  i  i.it  the  instructors  in  the  radar  observer  training  program  ditFered 
;;t«  .tly  with  respect  to  the  cpialitie'  of  motivation  and  instructional 
ability  upon  which  they  were  rated.  This  conclusion  agrees  with  the 
•  'b- .  rva I  ion  of  Radar  1‘rojcct  personnel  who  were  enrolled  in  the 
course.  An  inference  of  this  is  that,  under  similar  circumstances  in 
the  future,  scientific  instructor  selection  would  yield  laroo  returns  in 
""I  i  oveitieiit  of  instruct  ion. 
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While  1 1  if  Radar  Project  made  many  informal  contribution" 
wartime  training  pr oerdnro.  time  did  not  permit  the  eoiidiiet  of  :u,v 
formal  trainin''  research.  I  IoWever.  since  projerl  personnel  (level.  (p,.,| 
considerable  familiarity  with  the  training  rnrrientnin.  the  followinir 
suggestion"  are  made  for  possible  post  .war  research. 

A  training  proldem  frequently  eneoimtered  in  radar  observer  train- 
ini'  eenter.-'  around  the  importance  to  learning  of  student  understand¬ 
ing  of  the  operation"  being  performed.  A  clear  example  of  this  ari-e" 
in  tidal  ion  to  instruction  in  the  adjustment  of  control"  on  the  radar 
equipment.  It  is  possible,  at  one  extreme,  to  teach  nse  of  the  controls 
almost  entirely  in  terms  of  the  elVrrt  they  produce  on  various  meters 
and  oscilloscope  screens.  At  the  other  extreme,  considerable  explana¬ 
tion  can  he  made  of  the  reasons  for  the  clFccts  produced  by  various 
adjustments.  In  wartime  instruction,  opinion  and  practice  on  this 
point  varied  considerably.  It  is  probable  that  guiding  principles 
could  he  worked  out  by  experimental  comparisons  of  curricula  which 
diller  in  the  extent  to  which  technical  understanding  of  the  equipment 
is  stressed. 

Another  point  at  which  the  value  of  technical  understanding  is  in 
doubt  is  in  radar  bombing  instruction.  A  series  of  procedures  with 
appropriate  qualifications  may  he  worked  uni  to  govern  the  radar 
observer's  behavior  during  the  bomb  run.  The  elTect  of  limitin''  in¬ 
struction  to  these  procedures  is  n  matter  upon  which  there  is  consider¬ 
able  disagreement.  How  much  an  understanding  of  the  theory  of 
bombing  and  the  mechanics  of  the  bombsight  speeds  learning,  con¬ 
tributes  to  accuracy,  improves  the  handling  of  emergency  situations— 
all  are  questions  calling  for  experimental  investigation. 

A  second  general  problem  of  radar  training  has  to  do  with  the  most 
eliieient  media  for  presentation  of  instruction.  The  use  of  motion 
pictures  and  other  graphic  aids  is  advocated,  but  the  extent  to  which 
stall  techniques  facilitate  learning  remains  undetermined.  It  is  possi¬ 
ble  that  graphic  aids  sin  h  as  motion  pictures  can  aid  instruction  in  two 
respects:  (</)  by  increasing  classroom  in  teres1,  mid,  thus,  the  motivation 
of  the  "Indent  to  learn,  and  (b)  by  enlarging  the  scope  and  vai  ictv 
of  material  presented.  An  example  of  a  potentially  profitable  applica¬ 
tion  of  graphic  aids  may  be  seen  in  the  chi"sroom  teaching  of  radar 
navigation.  Here  the  Use  of  actual  or  artificial  scope  photograph-  in 
simulated  missions  might  afford  a  closer  approximation  to  aerial 
missions  in  the  classroom  than  is  accomplished  through  the  problein- 
soh  inn  materials  currently  used.  In  addition,  the  effectiveness  of 
motion  pictures  as  an  aid  to  briefing  for  aerial  missions  should  la? 
invest  igatod. 
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A  r  i  if-  of  |  x  >""  ii  ilc  research  |  hi  <1  ildi  i  -  relate  t  it  ee  i  mi  mi  i  \'  n  i  t  ra  i  m  n  o 
1  i  11  :<  .  A  i  t)iielii"iiiii  f  n  >ni  tile  extended  t  la  1 1 1  i  1 1  experiment  «*•  iii<  1 1:1- 1  I 
I , \  'ir  National  Defen-e  Ke  rarcli  Committee  is  tlial  student  radar 
uh-  \ef"  reiptire  approximately  I  U»  hoiir~  nf  aerial  practice  In  reiteli 
pea  >  bombin''  cllicieucy.  It  is  probable  that  in  the  wartime  radar 
U;i  i ■  i 1 1 i-T  program  mule  time  than  this  wnnld  have  heeii  needed,  -inn- 
the  NDKC  experiment  iva«  eimdin  ted  under  condition"  specially 
f:i \ •  ‘ii n*r  learning  and  atypical  of  the  radar  trainin''  program  in 
I'etiera I.  I  hi."  exorbitant  price  in  aerial  training  time  raises  re»eareh 
i|lie-lii*U>  a>  I**  whether  the  time  devoted  to  other  feature."  of  the 
ruin'C  could  he  profitably  revised  in  order  to  reduce  the  nnniher  of 
In m r."  rei|iiired  for  a  "Indent  to  reach  maximum  ellieienev  in  the  air. 

One  specific  source  of  economy  would  he  to  attempt  to  govern  the 
length  of  d  i  tie  rent  pa  ris  of  the  oromni  school  curriculum  in  accordiince 
w  ith  their  rout  rilmt  ion."  to  the  achievement  of  peak  aerial  proficiency. 
Ili i >  problem  is  essent ially  one  of  discoveriii"  optimum  training 
period."  in  tin*  various  a teas  of  "round  school  iiist  ruction.  A  potent ial 
"indy  of  "iipei'onie  trainer  i 1 1 > t ruction  may  serve  us  an  illit"t ration. 
It  i."  possible  that  n  Militant  ial  increase  in  supersonic  training  heyond 
that  prov  ided  for  in  the  vva  rl  line  enrrienl mu  would  "Iiorten  the  amount 
of  aerial  practice  needed.  On  the  other  hand,  it  is  equally  possible 
that,  heyond  a  certain  point  in  supersonic  trainer  instruction,  addi¬ 
tional  practice  would  interfere  with  the  development  of  aerial 
proficiency. 

1  *’  ‘.h  a  |  i"  the  mo.-l  fundamental  prohlem  encountered  in  training 
i"  the  proper  organization  of  instruction  in  the  component  skills  of 
the  radar  observer  s  task.  Instruction  may  proceed  ahniLT  either  of 
two  lines — or  it  may  follow  some  middle  course.  It  is  po"sih!e,  on 
the  one  hand,  to  assume  that  the  radar  observer."  task  consists  of 
badcally  discrete  skills,  and  that  the  integration  of  these  skills  should 
he  attempted  only  after  each  Inis  been  nuMcrcd  separately.  Thus, 
"rparate  courses  would  he  "iven  in  the  major  phase"  of  the  radar  oh- 
seiier‘s  job:  radar  navigation,  scope  interpretation,  "id  operation, 
rad  ir  bomhitio.  and  so  forth.  Another  view,  on  the  other  hand.  look" 
up* *n  the  radar  ol»~crver*s  task  a"  primarily  a  complex  of  "kill",  all  of 
wh  * ■  h  are  so  interrelated  functionally  as  to  make  separate  imtructioii 
in  i-ach  unrealistic  and  uneconomical.  In  this  view,  ellieient  leatniii" 
i'  ided  most  bv  ofcauizinc  all  learning  aroinid  the  intecratmn  of 
"ki  I",  'rims,  aerial  and  trainer  practice  would  coii'titute  the  m.ijoi 
I’ha-e.s  of  the  curriculum  with  instruction  in  component  "kill"  "itch  us 
iia-.  ication.  Iiombiii".  set  operation,  and  scope  iutei  pn  tation  taking 
pla- e  w  ithin  the  context  of  the  total  tusk.  A  re-cun  h  answer  "lioiild 
Ik-  -oiiolit  as  to  the  correct  compromise  between  the-e  two  views. 
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Selection  Tests  A  aiidaled  Airainst 

P 

Success  in  ITularObserverTrainim''' 


1  1 1 1  cf  -fleet  ii hi  liatterie-  i  t'i i 1 it  ino  |s  printed  and  p-vehoinotor 
t>  i-  "t  i  e  validated  a^ani-t  .-ticee.-.-  m  radar  oli-erver  training.  Thc-e 
aie:  tlie  Aircrew  (.‘la--i  lieal  ion  Battery  ( November  1  *. 1 1 '  i ) ,  the  Kadar 
()b-*r\  er  Selection  Hatterv,  anil  the  Kxperiniriit  al  Hattefv.  I)e>- 
i*i  i  1 1 1 1 1 >1 1 -  of  I c - 1 -  in  tln*.-«*  hatterie-,  together  with  diagram-  showing 
tlf  mmpiMtiiin  of  the  three  aircrew  -penally  .-tanine-,  are  [>iv  ented 
in  this  appendix. 

MIBKIAN  CIASSIMCATION  H  VTTKItY  (NON  lAIHKK  l«>i:j) 

Printed  Tests: 

Te.-t  :  Blue, apical  Data  Blank,  (ThiOi'I). 

I  )e-eript  ion  :  Thi-  te.-t  contain-  items  of  personal  lii-torv  which  are 
related  t o  .-neee.—  in  pilot  and  navigator  t raining.  It  eon-i-t  -  ef  items 

•  oneeinino  the  imli  v  •  *  1 1 1  a  1  *  -  educational  hi-torv.  work  experience, 
-poll-  and  hoi d i v  participation,  place  of  hirtli.  parentage,  etc.  The 
iti  in-  in  thi-  te-t  were  -elected  from  a  uni'  h  larger  iiiiinlier  which  had 
I  C**n  tried  out  experimental!  \  ill  previoli-  form-  of  the  te-t  (( ’  Kli'  J  A 

ai  d  CTT'diiB). 

N nnilier  of  items,  155 
1  i me  limit,  'Jo  minutes 
S'oi 1  no  formula  :  U  \Y  *-*j0 

'  *.t  1 1  •  i  1 1  •)  |i\  1 1  \  1 1 1 . 1 1 1  ID  II. -r  w  till  I  lit'  :i  ** -l- 1  ,t  ii*  ••  ..f  Sjl.  \  1  f  i  •  •  1  S  ArifM  » i  •  1  '-.:i 

'•  n  11  IliMnrf. 

I  -  •  \  >  r  1  |o*in!**  In  1 1 1  •-  r*  r  1  r*  f*  r.-in  •  )tw  I  ••  ••  1 1  iii.i  !•»  .  •  •  i  •  I  •  -  »*f  l*-f*  •ml  r 
f  1 1  •  ••  •  |n  i  U  w  I*  :•  1 1  u  i  rv  I  *i  Ii.»  v  ••  li.it  I  in  1  ip]  i  <1  I  It  :i  tv  » ]  *|»*  f :  - 1 1  \  U  |’i  I*  r  1  •  •  I  1  I  •  ,* 

"  r  •  5  •••  i  l  i!  f  *.  « 1.  *  I  • !  i  •  1  lit. i?  I  li«  ini  I*  I  l.iN  W  *T  •*  ii*it  "f  *nl!i.  ••  fit  t.»l  hd.  :  t 

.  *  i  - .  i  •  ••  ,‘..r  ili*  i  ■  •  i  *  - 1  *  1 .  r  .i  I  •  I  ••  |».j  « •  •  I  li**ir  j-n  ■-»  r**(iiir.'l  l’»  i  •  ■  n »  1 1 1 1  •  r  1  s» 

•  ,  . . J . ^  ,.f  ||„.  f.vi.K  v\  1 1 1  1) 1 1 •  1  ||..|||  «.||  111**  In  111.-  mr  *  ..f  III.-  Mr  n  ID  *1 

'«  hi'.,  Arm*  Mr  fun  •  1>  :ti.*l  1 '  -  >  •  *  ^  1 .  .  1 1  >  •  •  1 1 . .  n .  t ‘I'D  .*  .-f  ID. 

v  .  .  .*i  rl»r«.  ,\.\I  I  r.tlnli.J  »  ..inut/i  fj'l.  I'.., :  h«.l  »D-  l'i»  M.  I -t  Sin*.  ID  ft-du  wld.li 

•  D.i  t  Dm  In  1 1  •  i ;  1 1 : 1 1 1  \  n.:t\  !•••  in. I'D*  .*i  r«*  "ii  Ml.-  In  Mi*  1  *«>}  •  Ii*  *1  •  “  •»  1  S«-.  I  !••»».  <>!-••  *•( 

f '  ii  • . .  1 1  1 1 .  j  •  I  ■  |  it. i  r f  i  -  A  \  I  ’  Tr:ili*ln£  <  *M.ini.in«l,  l?.i  rV***l  i  h  l«-M.  I-* 
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I  v  i  ■  I  i :  *  I » 1 1  i  t  :  Pilot  Key.  r  u.-iy  Navigator  KfV.  ;  n.  pi. 

1  if  Hi-  I  il)l  ;i  1 1 1  •  ■  1 1  ill,  a  lf-t  l'l  •  1 1  •  - 1  liii'i'  with  ;(  I  inn*  interval  uf  :i  |  >[  >1  •  i.\  i  - 

inal  r| y  days.  N  711.* 

Sample  item  :  During  in"-l  of  \  mu li  ft  you  liav v  lived  : 

A.  Ilia  large  eitv  (over  1  f  n  l.i :( ):  l ). 

IV  In  a  i-ily  (  lo.iiiM)  m 

(’.  In  a  .-ln;i 1 1  luwii  (  1. . .  in  lu.iMiii). 

I).  In  a  vf  ry  -mall  town  i  under  1  '»**<») . 

K.  1 11  t  hr  roil  lit  I  V. 

IC-i  :  Dial  Ki  a 1 1  i i \yr.  (  IY.gjA. 

De.-i ripl  ion  :  'l’l I i-  t »•.-!  n  < ] 1 1 i n*-  tin*  -nl >j«-»  i  to  read  \arioii-  l:ds 
« 1 1 1  i «  kly  and  accurately.  It  eoii-i-t-  of  1"  |  •  a  yr  i  *  - .  i  a-  li  page  containing 
7  dial-'.  The  scales  on  tlir  d i Hi- 1«- 1 1 1  dial-  vary  in  -ize.  angular  range, 
and  unit.-  of  mea.-ure.  I  In*  j i -d !«•  of  i-ai  li  dial  | ><> 1 1 1 1 -  to  some  value. 
Ill-low  tlir  hank  of  dial-  are  a  number  of  live-choirc  item-  consisting 
of  (In'  label  of  an  in.-l  nniM'iil  (ainnci  v.-.  aliunde.  etc.)  and  five  alter¬ 
natives  foi-  tin1  leading.  The  task  -  to  liiai  die  dial,  read  it.  and  tlu-n 
find  I  lie  correct  answer  among  the  live  alternatives.  This  te.-t  i-  printed 
as  the  lir.-t  part  of  a  booklet  which  also  contains  Table  Readme, 

CPr.JlA. 

Number  of  items,  7*7. 

Time  limit, minutes. 

Scoring  formula  :  ( 'on  il  lined  with  Table  Reading.  Cl'ikil  A  ;  R  —  W 

Reliability:  /  (1.7(1.  C’orrelat ion  between  two  separately  turn'd 

halves  corrected  for  length.  X=  1.107.* 

Sample  items:  See  Figure  A.l. 

Te.-t  :  Table  Reading.  ('P0*J1  A. 

Description:  This  te.-t  is  printed  in  a  booklet  with  Dial  Reading 
Cl’tij-JA,  and  consists  of  two  parts  which  are  timed  separately. 

Part  1  consists  of  a  large  bivariate  table.  The  subject  is  given  a 
pair  of  marginal  values,  and  he  must  ti ml  the  cinrv  in  the  ImhIv  of  the 
table  which  is  directly  below  one  value  and  horizontally  in  line  with 
the  other. 

Part  1 1  consists  of  a  set  of  four  table-,  each  of  which  has  a  pair  of 
entries  for  each  of  the  three  values  of  one  variable,  and  1  •  >  entries  for 
a  second  variable.  The  subject  must  determine  which  of  the  f*ur 
(aides  to  enter,  tind  the  correct  column  and  row  in  that  table,  a  ad 
ti lm  1 1  v  -elect  the  correct  pair  of  nmnlvrs. 

Number  of  items;  Part  1.  4d:  part  II.  b'k 

Time  limit  :  Part  1.  S  minutes:  part  II.  7  minutes. 

Scoring  formula:  Combined  with  Dial  Reading.  (TY.gg  V  :  R  —  \\  i 

•  (iuUr»r>1,  J  l'.  .■"'it  Ijivj.  J.  I.  r<f*  PrtnTri  <J  HM>Hru(pi/«  Ufl*.  AAP  kvU'h'0 
)im  l'lMcrum  ro.,'*tvh  rvis'rts,  Nn.  ,Y  Wn-hiiirii-n :  Cfi *-niu»*-ul  1‘riuiiuc  Of-rr. 

1IUT.  CliAiUor  'it. 

’  I  hi. I  .  .  lmjil'T  trt. 
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altitude 


VOLTME  lr 


FUEL-AIR 

RATIO 


AMPERES 


I.  K.  V.  M. 
II,  Amperes. 
III.  Altitude. 
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correct  ones. 


lo-liabilit  \  :  /  it. >1,  Correlation  between  i  wo  i  ateiv  limed 

]i  |\ e-  ci 'l  ied i  ( i  fur  length.  X  1 1  * »7. * 

S : 1 1 » 1 1  *  1  < •  i 1 1 * 1 1 1 "  :  See  Figure  A.l. 

I'e.'l  :  (ielliTal  I  ll  f  ( •  11 1 1 : 1 1  i<  >n.  ( '  K.‘«l  I.Mv 

1  >i  ■-(  •  ri  I  *  t  inn  :  !  Iii>  te.-t  contain-  item-  ha-ed  mi  know  led^c  nf  air- 
,  ifl.  avialimi  technique.  automobile  dlivino.  mechanic-.  ale)  [  k  >  1 1  ^ 
;l ; , i )  lmlibieS. 

Xuiiilter  nf  item- :  Part  I.J5;  pari  1  pa rl  III.  15. 

lime  limit:  Pari  I,  in  minutes;  pari  II.  1J  minute-;  pari  III,  ll 
minutes. 

Sidling  formula:  ]{— W  -I. 

Reliability  :  /•  U.Si.  Correlation  between  mill  ami  even  ilrm-  roi  - 
icrteil  for  Icnolh.  N  1 .1)01). 

Sample  ilein:  \\  liielt  one  of  the  following  i-  nm-t  eoimuoiilv  used 
lo  l  rain  pilot.-  mi  the  ground  ? 

A.  The  Waco  Trainer 
H.  The  Ryan  'Trainer 
C.  The  Fairchild  Trainer 
I).  'The  Whit  *  Trainer 
H.  The  Link  Trainer 

'Test:  Instrument  '  '<  'luprehensiou  I,  ClMoR. 

1  )e.-rriptioii :  This  test  requires  the  siihjeel  to  interpret  readings 
of  six  aircraft  instrument::  and  to  relate  the  instrument  readings  to 
verbal  descriptions  of  an  aircraft's  performance.  'The  test  explains 
the  function  of  the  six  instruments  lo  the  subject  :  Altimeter,  artificial 
horizon,  compass,  rate  of  climb  indicator,  air  speed  meter,  and  turn 
1  ank  indicator. 

In  each  item  the  subject  is  presented  with  a  drawing  of  the  six  in¬ 
struments  with  tlm  pointers  showing  reading-  and  live  verbal  state- 
incuts  describin';  tin*  action  of  the  plane,  such  as  “Flyiuo  level  at  •.!<•<) 
in.  p.  h.,  straight  and  ltnhauked,  headed  due  south,  •'ainiii;'  altitude  at 
!•, Mil)  feet.’’  The  subject  nm-t  choose  the  one  of  live  dc-cnplimi-  which 
fits  the  readings  of  the  six  instruments. 

No.  of  items;  15. 

Time  limit  ;  1*2  minutes. 

Scoring  formula:  20  —  (It  —  W/l). 

Reliability:  /'  -  O.M.  Correliitiou  between  odd  and  even  item- 
orreeted  for  length.  X-5i)0.s 
Sample  Item : 

A.  I'hitiK  lev  i*l  lit  '_'( M l  in.  p.  li.,  stratulil  amt  unliaiikcd,  leaded  dm*  smilli. 
it  lilt  iik  altitude  at  l*.s'c0  feet. 

It.  l-'hine  level  nt  JoO  in.  p.  li.,  slralitlit  nml  nnti;ie.kcit.  leaded  due  -".ih, 
'»lnc  allllilde  al  .1.000  fiH’t, 

(.'.  Klvllltf  level  III  L’oo  III.  J|.  ll.,  hlrnljilll  II III  1  Ullt'.lllked.  l|e:lde<l  due  .’'•'lllll. 
' 1 1 1 1 i 1 1 1 .* I i I > 1 1 1 rr  allllilde  ill  d.OOU  feet. 
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I).  F1.v1.ik  lovol  n(  _’(H»  in.  j).  h.,  .stnilKlit  nml  hnukod  to  loft,  bonded  due  norlli, 
mnliitiiliilliK  iillllildo  nt  '1,000  fi*ot. 

K.  FIyIiik  lovol  nt  L’lKl  in.  j».  li.,  turning  pro|iorly  In  loft,  with  30°  lunik,  innlu- 
t;ii n!. ik  nlllriido,  linidoil  duo  north  nt  1,000  foot. 

DcmiIjiIIoii  ( ’  Is  the  cor  loot  miswor.  F.xnniliio  thoso  dinls  iiitnln  mid  elioek  thorn 
onrofiilly  wliilo  dosei'lptlmi  C  1h  roml  to  you. 


I  e-t  ■  I  ll.-t  t'llllii'lil  (  i  ■  1 1 1 1  >  1  rlicii'inll  II.  I  *  1 1 » 1 1»  I  i. 

I  )i‘  —  i  I  I|>1  1**11  .  Fa  el  I  Until  e(  1 1 1  -  I  "I  "  (if  ;i  li'|  ili'd  ||  ;|  |  |ii|)  nf  C  .III  pa 

el  all  iliinl  horizon  and  live  pi*  t  mi  -  nf  ;,i,  a  i  r«-r:,  fl  in  ililfereiil  at 
i  t  nde.-  of  lli^lil .  I  lie  .-ill ijrci  mii'I  indicate  u  Inch  | ni  l  n t v  inn e'-| mud- 
i  .  tin*  tvadiiif'*.  nil  the  rmnpa-,  and  aitilicial  hmizmi.  The  picture  is 
i  •  >i  i  >  l  •  1 1  *  i«  *»  1  In  i  i  a  v  i  •  I  >i*i  *  1 1  I  a  Ur  n  f  f*  »i  1 1  I  In*  sontli  and  mi  a  level  with  tin* 
ircra  ft.  so  that  ah  aircraft  hoadiii"  -mu  h  and  living  >t  rai^lil  and  level 
i  ■  seen  exact  ly  head  mi. 

Xntnher  nf  items,  (il). 

Time  limit .  In  minntes. 

Scnrii!^  fnrtimla  :  R  \Y  l. 

Reliability  :  /•  tl.Un.  Cm  relat  imi  lietvveeil  ndd  and  even  iti'ins  enf- 
rerted  fur  length.  X-  hlMI.'1 

Sample  item:  See  Figure  A. a. 

1  )i leia Inns :  in  eneli  nf  tin*  prnhtciiis  la  | i;i rt  11  \ini  will  In'  clvcii  ii  picture  nf  ii 
single  plane  In  live  illlTel'enl  push  juiis.  At  the  left  nf  I  lie  picture  vnli  will  lie 
shown  tu'n  dials.  an  artitleial  Imil/.  >n  anil  a  compass.  Veil  are  In  clnm-c  the 
I'li'llimi  nf  the  plane  wlilcti  agrees  with  the  readings  on  ihese  dials.  In  reaillni; 

I  lie  dials,  reineinher  ilia  I  ,vmi  me  al  the  mnirnls  nf  the  plane.  Iliukhin  forward. 

I.nnkiiiit  at  the  picture,  a  plane  hcadim;  away  fmni  ,\nii  is  ip'lnu  north. 

rialles  tlylll}'  Snlltll  Will  Ii . .  directly  Inward  Jnll,  white  those  Koilij;  In 

\niir  ri^hi  w  III  he  heaiieil  east,  ii  ml  these  In  ymir  left,  headed  west. 

I’l'nhleiiis  A  and  It  are  samples,  Kxainhic  die  readings  nil  the  dials  at  the  left 
nf  tlie  pictures  in  Pl'nldeill  A.  New  lonk  at  tin*  live  pns|||nlis  nf  the  plane  and 
select  the  pns’dlnli  wlllell  Is  correct  aeemdini;  In  the  readings  nil  llie-e  dints. 

According  tn  I  In*  dials,  the  plane  Is  living  level  and  nnhankisl  and  Is  headed 
dlle  wesl,  ,\n|e  I  hat  tile  plane  at  |nsl(lnll  1)  Is  the  only  nlie  wlllell  Is  correct 
lor  all  these  readings.  N’oiice  I  hat  |Hisitlon  It  Is  i  orris  I  in  every  rc'pe<  t 
i  \eepl  that  I  he  plane  Is  IlyilijJ  soil  111.  position  ('  Would  also  he  enrreel  except 
that  the  plane  Is  living  north,  ttciiicinhcr  that  every  rcadluu  oil  t tie  dials 
inllst  he  cheeked  ill  order  In  determine  the  enrreel  push  lull. 

l’est:  Mathematics  A,  (T7ii'JF  ((irtieral  mat  hematics). 

I  )eseripl  'toil :  'I’ltis  I  est  is  de'i”ttcd  tu  mcii'itiv  know  led^e  of  algebra 
i ml  elemctitary  I  rim up unet ry.  The  stihjeei  is  rei|ttire<l  to  solve  eqitu- 
1  imis  and  I  rinoumiiel  rie  problems. 

Xumher  of  items,  da. 
rime  limit ,  *2  a  minutes. 

Seiiriuo  formula  :  2R  -  W.  2. 

Keliahilit v :  r-  U.th’h  (’orrelatiou  hetvveeu  odd  and  even  items  n»r- 
: eeteil  for  length.  X  1,1  Mil).1 


*  /  Mil. 

1  Iteni'jirrli  lliiltrl  In  tt  IS.  |‘.)  rli. 'Ii'idi  id  Si'll.. a.  oilier  ..f  <t..-  S  •of.'i..  llcol.piitlrr*. 
A !•*  Tridiilin;  Cnmnuiml. 


2H7 


Sample  item:  If  a  ami  h  1.  the  mnm-rical  value  of  a1  h'  • 

(  i  h)’« 

A.  !>S. 

B.  7S. 

('.  tic.. 

1).  IS. 

E.  :u. 

1  c-t  :  Mathematics  B.  Cl'JtMlC  ( arit lunet ie  reasoning). 

1  )e.-(  ri j.t mu  :  I  lii>  test  requires  the  solution  of  arithmetic  problems 
expressed  in  verbal  form. 

Number  of  items,  .‘50. 

Time  limit,  B.*»  minutes. 

Scoring  formula:  2lt  — W/2. 

Reliability :  r  -O.S-t.  Correlation  between  odd  ami  even  items  cor¬ 
rected  for  length.  X  —  1 ,( K">0.M 

Sample  item:  If  plane  A  can  lly  l.*.t)  miles  while  plane  B  E  dying 
100  miles,  lmw  many  miles  ran  plane  A  lly  while  plane  B  is  Hying 
2.10  miles? 

A.  275  miles. 

B.  .*’>00  miles. 

C.  M0  miles. 

D.  .’150  miles. 

E.  1575  miles. 

Test:  Mcehanieal  Principles,  ClOO.'lB. 

Description:  This  is  n  test  of  the  ability  to  understand  mechanical 
forees  and  movements.  It  unhides  items  covering  gear  systems, 
mechanical  movements,  the  principles  underlying  physical  phenomena, 
and  a  number  of  items  concerning  levers,  propellers,  pulleys,  etc. 

Number  of  items,  10. 

rime  limit,  20  minutes. 

Scoring  formula:  U— W/2. 

Reliability:  /•  O.S.'l.  Correlation  between  odd  and  even  items  cor- 

i cried  for  length.  X  -1,000. 

Sample  item:  See  Figure  A.O. 

Test :  Reading  Comprehension,  CRU  HI. 

Description:  This  test  is  made  up  of  S  paragraphs,  each  of  which 
•  onsi-ls  of  250  to  1100  words  in  length.  The  paragraphs  deal  with 
technical  topics  including  compass  variation,  dark  adaptation,  com¬ 
pass  compensation,  Mercator  projection,  the  air  sped  meter,  conip.i-s 
hearing  and  Bridgmans  operational  concepts.  Each  paragraph  is 
followed  by  several  live-choice  questions. 

*  Oiillfi  r.l,  J.  I*,  mi"!  I -in- jr.  J.  I.  t>h..  <■;».  cff.,  rlin|>lrr  7. 


Xiimher  of  items  :;r». 

Time  limit ,  •'in  minutes. 

Scoring  formula  :  'Jit  \V,  2. 

Reliability  :  /■  * >.>>.>.  (’orrel;il  ion  between  odd  mnl  even  item>  .)•_ 

reetetl  for  length.  X  1 .000. 


Sample  item :  As  the  eyes  heroine  dark  adapted.  the  relative  bright* 
l less  of  the  » 1  i ITe rent  colors  in  the  spectrum  changes.  'The  point  of 
maximum  brightness  in  an  intense  prismatic  spertrmn  is  in  the  yellow 
for  the  l i «rh t  -adapted  eye.  As  the  brightness  of  the  total  speetnnn  is 
lessened  and  the  rye  becomes  dark-adapted,  this  point  gradually  shifts 
into  the  green.  1’hat  is,  the  shift  in  brightness  as  intensity  decreases 
is  from  the  long  wave  lengths  in  the  red  end  of  the  spectrum  toward 
the  short  wn\e  lengths  in  the  blue  end  of  the  spectrum.  If  tin*  eye  is 
thoroughly  dark-adapted  and  the  intensity  of  the  spectrum  is  further 
diminished,  the  spectrum  becomes  colorless  just  before  it  becomes 
invisible.  In  the  colorless  condition  the  spectrum  still  differs  in 
brightness  at  different  points. 

1(1.  'I’lie  strain  on  the  guy  wire  is — 

A.  greater  if  attached  in  position  X. 

B.  greater  if  attached  in  position  Y. 

(’.  the  same  if  attached  in  either  position  X  or  V. 

Which  of  the  following  would  have  the  shortest  wave  length  in  an 
int  ease  sped  rum  ( 

A.  lied. 

B.  Yellow. 

C\  Yellow-green. 

I).  (Iree n. 

E.  Blue. 
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I  e~t  :  Xpal  i;i!  ( hiental  ion  I.  (T.Vi|  It. 

I  )i-it i | *t  iuu  :  Tlii>  i  -  :i  |c-t  nf  ilu.  v  t<.  ;iir  'in:i|]  i«»n-  nf  m 

ri;il  pln*lu^i  ;i|'li  w  i: iiiu  a  huner  picture.  Tin*  tc-t  .  nn-i-t-  nf  ,  i^lu 
i i'll1'  :i fi'i a  1  l»lu»t | >li-  ;  >i\  exempts  hit  In  l .«*  located  in  ,a<h. 

X  umber  of  hems,  is. 
rime  limit,  n  minutes. 

S.  "linn  formula:  R  \V  in  ( ‘la  — i  In  at  i*  >n  I’.aitciy:  total  ri"ht 
ihidar  Observer  Select  mn  ltaiterv. 

Kel  lal  ii  1  it  V  :  /'  ^  « .  (  orrelat  mu  lietweeii  mill  ami  e\cii  ilclii'  nil 

reeled  for  length.  X  l.OOtl. 

Sample  item:  See  lupine  A. 7. 

Te»t  :  Spat i ;il  Orientation  II.  ClViUoIt. 

1  Ji'M'iipt  ion  :  1  Ins  is  a  lest  ot  the  a  hi]  uy  to  locale  an  area  on  a  map 
correspondin';  to  a  sect  ion  of  an  aerial  photograph.  The  te»l  consists 
of  12  panes,  each  of  wliich  contains  a  part  of  a  ~<:uidard  aviation  map 
in  color,  ami  aerial  photos  of  four  small  areas  within  the  area  eovere<l 
by  the  map.  The  subject  must  determine  in  which  section  of  the 
map  lies  the  area  covered  hv  the  photo. 

Xninher  of  items.  IS. 

'l'inie  limit. minutes  per  part. 

Seorinn  formula:  H  —  'W 

Keliahility:  /•  it.sO.  (’’orrelat ion  lietweeu  odd  and  even  items  cor- 
reeleil  for  length.  X-::l.t)Ut). 

Sample  items:  See  l'inniv  A.S. 

Ps\  (  hoinotor  Tests 

Te-t  :  Complex  ('oor<  limit ion.  ( 'M7t>l  A. 

1  )eseript ion :  In  this  test,  the  subject  operates  controls  similar  to 
those  imcd  in  an  aircraft  in  1 1  i h t .  A  stick,  as  in  an  aircraft,  can  he 
moved  forward,  backward,  and  laterally.  1  lie  feet  operate  pedals 
-imilar  to  the  rudder  emit  nils  in  a  plane. 

In  fl  out  of  the  subject  J*>  a  stimulus  panel  mi  u  1 1 i « •  1 1  I  here  ale  l  lu  re 
rows  of  red  lights  and  three  correspnudiiin  rows  of  pMeeu  lights.  A 
pattern  of  red  linht s.  one  in  each  row.  is  prc'cnl ed  to  l  lie  subject.  Ills 
! ask  is  to  move  his  emit  mis  so  as  to  t  urn  on  the  nrccii  linht  correspond- 
inn  to  each  of  llic  red  lights.  By  movinn  liis  st  ick  from  lefi  lorinlil. 
he  can  control  the  «jrecn  linhts  in  the  top  horizontal  row;  hv  new  in;; 
his  st  ick  backward  or  forward,  lie  can  coni  ml  the  linhts  in  the  \  til  if  a  I 
row;  hv  niovinn  the  pedals  he  can  control  the  linhts  in  the  bntinm 
horizontal  row.  As  soon  us  each  of  the  three  red  linhts  is  matched 
hv  the  rorrcspolidilin  »rccii  li^ht,  il  new  s,-t  (>f  red  lil/hls  is  pieo-ntcd. 
The  subject's  task  is  to  match  as  many  sets  ( if  linht  >  as  possible  v.  it  bin 
a  specified  period  of  t  ime. 
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Xumberof  items:  One  tiM  period. 

Time  limit,  M  minutes. 

Scoring  formula :  X  umlier  of  pat  tern.-'  correct  1  y  mat  el  ml  liming  i  i-t 
period. 

Reliability :  r  0.1)1.  Correlation  between  odd  ami  even  trials  mr- 
reeled  for  length.  X  l*2a. 

Apparatus : 

I  '  ”  ~  ... - 


i 

v. 


Second  Sample  Item. 

Km.  A. 11.  Complex  i ’oonltun  1  ten  Test,  ('.M701A, 

Test  :  DiM  i  iminat  ion  React  ion  'l  ime.  C’Rblll). 

Description:  In  this  test,  the  subject  must  react  to  the  relative 
position  of  a  red  and  green  light  on  a  st  imulus  panel.  Four  lights  are 
arranged  in  the  corners  of  a  square.  The  upper  left  light  and  the 
lower  right  light  are  reil.  'The  upper  right  light  and  the  lower  left 
light  arc  green.  A  w  hite  light  is  used  as  a  warning  signal ;  then  a  pair 
of  lights,  one  red  and  one  green,  are  illuminated.  One  of  four  toggle 
sw  itches  must  be  pushed,  depending  upon  w  hether  the  red  light  appears 
above,  below,  to  the  right,  or  to  the  left  of  the  green  light.  A  time 
•  lock  records  the  time  until  the  cored  toggle  switch  is  pushed;  the 
time  is  added  for  all  trials. 

X mnbcr  of  items,  N)  trials. 

l'ime  limit  :  None. 
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>n»riuj.^  formula  :  1  olal  t hue  neressarv  to  < *« xi i [ •  lt*t i*  M)  t rials. 
Krlia'niit y :  /*  (>..ss.  Conflation  between  otld  ami  even  trials  eor- 

i,  a  oil  for  length.  N  - 
Apparatus : 


f" 


i 

\ 

i 


Kim  UK  A.  1 0. — IKserliiUuutlon  reaellim  | Inn*  |e>(,  (TUIID. 


Test :  Kinder  Dexterity.  C.MllHA. 

Description.  Tue  <est  consists  of  a  form  board  (s  \  'Jl  x  1  inel»e>) 

•  ontainin*;  1^  sipiure  holes  1 V » o- i m*l» >  »:!«-<•» I  I •  ■>  i apart,  ami 
I**  pe*rs  i ni'lt  s«|uare  with  roiiu.l  tops  t  inch  in  diameter.  Tin* 
-uhjcct’.s  tusk  is  to  remove*  each  pejr  in  turn  with  his  rijdil  hand,  rotate 
i!  clockwisi  a  half  turn,  ami  reinsert  it  in  the  hole.  The  lower  ami 
i  pper  halves  of  each  pro  top  are  painted  uniformly  dilh  rent  to  aid  in 
••••irrminino  the  iiuiuher  eom*»*lly  turned. 

Number  of  items,  live  trials. 

Time  limit,  !i’i  seconds  per  trial. 

Scoring  formula  :  Total  iumiiIhm*  of  pegs  turned. 

Ucliahilily :  r  o.hd.  Correlation  between  odd  and  even  trials  cor- 
teeted  for  length.  N=I*J5. 

ApjKiruttis:  See  Kijjnre  A. 11. 

Test:  Ihitarv  Pursuit  with  Divided  Attention.  CIMlull. 

Den  t  ijition:  The  suhjeeTs  task  is  to  keep  the  point  of  a  stylus  in 

•  outuet.  with  a  metal  target  which  is  set  in  a  revolving  turntable,  and 
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I 


i 


i 


Fit. i  >; k  A  II.  Fiiint  i-  ilcxlecily  It  si,  (.'MlllJA. 


w  it  li  his  If  ft  h:mtl  to  press  down  ;il  :ill  t  hues  either  one  of  two  buttons 
<ni  ;i  box  to  t  lie  left  of  the  turret.  The  t  mutable  is  rotated  at  ;i  speed  of 
(in  r.  p.  in.  The  button  to  he  kept  down  depends  upon  whether  a  red  or 
a  green  bold  is  illuminated.  The  divided  attention  aspect  of  the  test 
is  the  requirement  that  the  subject  must  shift  from  one  button  to  the 
c  liter  each  t  hue  t  he  signal  light  eh  a  Hires.  He  receives  a  score  only  when 
the  st  vlus  is  in  contact  with  the  rot  at  in;:  target  and  t  he  correct  button 
is  pressed  down. 

Number  of  items:  Fixe  trials  without  divided  attention,  10  trials 
with  divided  attention. 

I  line  limit  ;  2<>  seconds  per  trial. 

Scoring  foi  inula:  Total ‘‘contact"  lime. 

Reliability  :  r  0.1M.  Correlation  between  odd  and  even  t  rials  cor¬ 
rected  for  length.  X  ITT 

Apparatus;  See  Figure  A. 12. 

Test  :  Kiulder  Control  Test,  CM120H. 

Description:  '1*1  u*  subject  is  seated  in  a  simulated  cockpit  which 
swings  on  a  pivot  and  is  mounted  on  a  heavy  base.  The  cockpit  can 
su  iiig  to  the  right  or  left  and  is  coot  rolled  by  pedals  similar  to  rudder 
controls  in  an  aircraft.  By  vary  in*:  the  pressure  on  the  pedals,  the 
subject  can  keep  the  cockpit  balanced  in  a  central  position.  Pushing 
the  light  pedal  forward  turns  the  cockpit  to  the  right  and  pushing  the 
left  peda  1  turns  it  to  the  left .  'The  subject ’s  task  is  to  keep  the  cockpit 
centered  that  a  sighting  bar  mounted  on  the  front  of  the  cockpit  is 
pointed  at  a  target. 
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l'n.riti:  A.l'J. —  Itulai.v  |iai.-uil  le.-i,  will)  iliviilnl  ;i 1 1 oil !•  •!>,  (TIloll. 

Items,  12  trials. 

rime  limit,  till  seconds  per  triii). 

Scorin';  formula:  Total  lime  on  target. 

Kelialiil ity  :  r  t>.!*2.  Correlation  between  odd  and  even  trials  cor¬ 
rected  for  length.  X  —  ton. 

Apparatus:  See  Figure  Add. 

lest:  Two-Hand  Coordination,  C'MHMA. 

Description:  The  subject  mn-t  control  the  movement  of  a  carriage 
and  maintain  contact  with  a  moving  target  disk.  On  tin1  carriage  an 
electric  contact  point  is  controlled  by  two  lathe-type  crank-.  F.acli 


■jo 


7o:;:ej7 


17 
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I 


l-'iii,  A.i:i.  - KmMrr  (Nmirui  Test,  CMIUOU. 


tTiinU  controls  movement  in  one  dim  linn  and  l»y  turning  both  cranks 
at  once  the  movement  of  the  contact  point  can  he  controlled  simulta¬ 
neously  in  both  direel  ions,  'bite  target,  is  moved  in  an  irregular  path 
hv  a  sNstem  of  cams  driven  hv  an  electric  motor.  The  target  makes 

•  •  *  r"» 

one  complete  revolution  a  minute.  'The  path  of  the  target  disk  fol¬ 
lows  an  irregular  pattern,  which  is  repented  every  fourth  trial. 

Items,  8  trials. 

Time  limit,  l  minute  per  trial. 

Scoring  formula  :  Length  of  time  contact  is  made. 


i  i  i 
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Uolr.dnlity  :  /'  <•>().  (  orrelat  ion  between  odd  and  own  l  rials  ror- 

i,  ii'd  for  length.  X—  I *2 . 

\  | > | >:i  1:1 1  ns  : 


Flu.  A.l-1.--T\vi>-I  laiiil  ( 'inirillii:i< Inn  To.-l,  (J.MIU1A. 


HADAK  OBSKHVKK  SKM.CTION  IJATTKKY 

l  ost  :  Coordinate  Heading,  CI’AMB. 

Doscript ion  :  This  tot  ron-d.-ts  of  a  circular  <jfraph  siundatin"  an 
oscilloscope  screen.  The  circle  is  oraduated  in  decree-  from  t) 1  to 
■TO0.  A  .-rale  epadnated  in  mile.-  runs  from  the  center  to  the  ode-  and 
concentric  circles  appear  at  10-mile  intervals  front  the  center.  Lo¬ 
cated  within  the  circle  are  tin- tics  lopro-eutin;;  target  return-  on  the 
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<>.'<•  i llo.-rope  screen.  1  lie  t :  1 k  fur  1 1  i trill  is  lu  determine  the  bearing 
and  ranges  of  ;i  da-h  line  from  the  center  ol  tin*  circle.  Of  t lit*  live 
choices  for  bearing  and  range  leadings,  only  1 1  if  last  dibits  are  iim  .  1 , 

Number  of  items.  So. 

Time  limit.  ‘JO  minutes. 

Scoring  formula:  Tot  a  1  right. 

Reliability:/-  •  >.!)•>.  Coi  relations  obtained  by  split-half  teelmi(|iie 
corrected  for  length.  X  -100.‘J 

Sample  items:  See  Figure  A.  la. 

Test:  Oscilloscope  Interpretation,  (T’SlTA. 

Description:  Ibis  test  approximates  the  rccognit ion  of  radar  oscil¬ 
loscope  signals  through  interference.  Three  standard  types  of  signals 
are  Used:  irregular  forms,  various  sized  cut  veil  lines,  and  various 
sized  blips.  The  test  consists  of  a  series  of  circles  and  rectangles,  each 
of  which  contains  a  number  of  one  of  the  three  signals.  The  task  is  to 
count  the  number  of  signals  which  are  hidden  bv  interfering  lines. 
An  answer  key  is  presented  to  enable  the  answer  to  be  entered  on  a 
standard  IBM  A-0  answer  sheet. 

Number  of  items,  DO. 

Time  limit,  lit)  minutes. 

Scoring  formula:  Total  right. 

Reliability :  /•- 1 ).'.)().  Correlation  obtained  by  split-half  technique 
corrected  for  length.  N-~-152.10 

'Test;  Pattern  Orientation,  CPSltiA. 

Description:  In  each  item,  a  pattern  of  circles  is  shown  in  a  square 
on  the  left  side  of  the  page.  In  a  huge  circle  on  the  right  side  of  the 
page  the  saute  pattern  is  rotated  and  shown  again  along  with  other 
circles.  The  task  is  to  identify  the  pattern.  In  order  to  do  this,  a 
cross  is  presented  in  the  first,  pattern,  while  in  the  second  pattern 
lettered  crosses  are  presented,  only  one  of  which  corresponds  to  the 
cross  in  the  first  pattern.  The  task  is  to  determine  which  of  the  five 
lettered  crosses  corresponds  to  the  cross  in  the  first  pattern. 

Number  of  items : 

Part  I:  21. 

Part  II:  24. 

Time  limit : 

Part  I :  It)  minutes. 

Part  II:  10  minutes. 

Scoring  formula  :  Total  right. 

Reliability:  /•-  0.71.  Correlation  between  part  I  and  part  II  cor¬ 
rected  for  length.  N— I70.n 

*  ositn  llfimrl  Nn.  isia.  Fnrmnl  .Mi'iiiormulmu  No.  1* r«*l  1  mliiii ry  Koporl  of  IIpxiiUi* 

from  0»i-llloM'o|n*  <  > | ■<■  rn I  ■  i r  Tout* 

'•MUtC  lnforiiiiil  Moonirn siilimi  No.  ‘J‘J.  'Jo  Miirih  IDl.'i.  Osrlllosi-o|n>  Iiilor|>rfUitlon  Tout 
for  tin*  Si’IitIImii  uf  Itinliir  0|M-rnli>m. 

11  Uro^ri  i-H  U i i r t  for  I'm lilliiilrr  1‘rojocl  of  AKItl)  No,  I,  S**j •  t .  1!>44,  j>.  0. 
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Sample  item : 


o 


O 


m 


O  O 


O 


o  N 


In  tin*  sifim note  t)u>  ] >n 1 1 itii  iif  two  small  ami  two  In rn«*  circles.  On  the 
right,  unit*  that  tin*  four  cross-liatehcil  circles  form  the  same  pattern.  The 
|iallern  has  been  rotated  so  tliat  now  the  two  small  circles  are  to  the  right 
rallier  than  to  the  ielt  of  the  two  large  circles. 

In  the  figure  at  the  right  there  are  live  crosses  lettered  A,  1$,  C,  I),  and  K.  Look 
at  the  cro-s  marked  It.  It  hears  Hie  same  relat loiishlp  to  the  tour  circles  as  does 
the  cross  X  In  the  square  on  the  left.  Therefore,  15  is  the  answer  to  problem  1. 
Indicate  that  11  Is  the  answer  to  problem  1  by  blackening  the  appropriate  space 
on  your  answer  sheet. 

KhiVUK  A. 17. — Pattern  Orientation  'Pest,  CTS10A. 

Test. :  Spatial  Orient  at  ion  I,  CPoOlB. 

Descript  ion :  This  test  is  described  in  the  preceding  section. 

Test  :  Radar  Preference  I. 

Descript  ion  :  See  chapter  10,  page  221. 
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4%  COORDINATION 

ROTARY  PURSUIT  (D  A) 
DISCRIMINATION  REACTION  TIME 


Fioi  iik  A.  2<>  -C"««m|>osil imi  of  pil"l  .-I :i ni m>.  I'l'ici'iii  eontr ilmlt-i!  by  e;uh  lest  to 
1  1m*  c-« mi i*« «-l l »•  jiiioi  ;i|M ii icli'  ruling.  Itnticry  of  1  NoycmiImt  11* IH. 


R  ml  nr  observer  ."I  a  nine  :  The  i-«  »i  1 1 1  >«  i-i  t  inn  of  1 1 1  i  -  st  nniiie  is  described 
ill  rli  ;i]  it  el-  10,  Ji:i”e  I.  :ts  derived  from  (lie  Ihidar  ( )b-cr\'or  Select  ion 
Battery.  Dntn  necessary  fur  the  preparation  of  n  dinornm  lire  nut 
available. 

KXPKKIMKNTAl.  BATTKHY 

Prin ted  Tests 

Test :  Aerial  Orientation,  CTVeJOA. 

I )e><  rij)t  ion  :  Each  item  consists  of  n  cockpit  view  of  the  terrain  over 
which  an  aircraft  is  living.  Tu  the  i  i^ht  of  the  view  are  live  pictures. 


•  ‘in'll ;*  1 1  o  w  i  1 1  ; i  i ilu in*  in  a  di  lie  rent  po.-ilion  over  the  t m  um.  The  ta-k 
is  to  111, ’ll  ell  t  111*  enek  pit  view  W ll  li  the  nil  T*,et  pi ;l lie  po- it  loll. 

Niiniher  of  items : 

Part  I:  :5U. 

Part  II:  -JM. 

'l  iiue  limit  : 

1  ’art  1  :  10  minutes. 

Part  II:  S  minutes. 

Scoring  foininly  :  'Two  scores  were  obtained,  total  ri<dit  and  total 
wrong. 

Reliability  :  /■  O.St).  ( ’orrelal  ion  between  part  I  and  part  IT  cor¬ 
rected  foi  length.  X— ‘H3. 


Tim  :  Area  Y  i-ua  I  i/a  t  i  t  >  t  ■ .  (  '  I *s  1  .'i  A . 

I  'i  l  ipl  imi  ;  I'.aeli  item  euii'i't<  of  two  geometric  drawing'  w  1 1 i .  J , 
lnlt'l  he  eomliineil  to  f  i  >1111  one  of  three  geometric  drawings  laheleil 
1  h  am  I  (  .  1  lie  t  ;t'k  i'  to  i  let  ermine  which  of  t  lie  laheleil  lira  wings  is 

the  re- lilt  of  fitting  the  first  tl r;i \vi u^r  together  properly. 

N ninher  of  items  : 

Part  I:  o<>. 

Part  II  :  :»(). 

'Fitne  limit : 

Part  1  :  7  minutes. 

Part  II:  7  minutes. 

Scoring  formula:  Two  snores,  were  obtained,  total  right  and  total 

wrung. 

Reliability  :  No  data  available. 

Sample  item : 


If  the  two  triangles  are  rotated,  they  would  fit  together  to  form  n 
Mpnin*.  Therefore,  the  answer  to  the  sample  problem  is  B. 

The  pirl are  ut  t lie  left  slmws  n  enck|iit  view  or  the  view  seen  hy  the  pilot  as 
he  looks  mil  over  Uie  nose  of  his  plane,  ian  li  of  the  tlve  pietures  nt  the  rinlit 
shows  a  plaite  in  a  ililTerent  position  over  ihr  eoast  line.  Your  tusk  Is  to  nniteli 
I  lie  eoekpll  view  with  the  eorreet  plane  position.  Not  let*  in  enrli  picture  that 
I  lie  roast  line  runs  tiirerlly  away  from  you  as  far  as  lln*  eye  ran  see.  Notice 
al'ii  1 1  la  I  ocean  is  on  your  rlfiht  alul  land  Is  on  your  left. 

1)  Is  i lie  eorreet  answer  to  prohieni  1.  You  can  tell  from  the  cockpit  view  at 
I  he  left  I  lint  liie  plane  Is  tlyliiK  level,  unhaiiked.  and  is  headed  directly  toward 
the  u i oi 1 1 1 1 n i 1 1 A  more  detailed  cxplaiml ion  will  follow,  lilaekeu  space  1> 
after  ileal  No.  1  now. 

In  order  to  s"lcd  the  piotu*  posillmi  from  the  eork|ilt  view  you  must  consider 
hank,  rlhnl.  or  dive,  and  ill  reel  hm  of  lll^ht. 

Test  ;  Air  Corps  Vocabulary,  1942. 

Description  :  This  is  a  speeded  test.  Each  item  is  a  word  followed 
hy  live  choices  of  synonyms. 

Number  of  items,  150. 

Time  limit,  15  minutes. 

S  oritig  formula  :  'Two  scores  were  obtained,  total  right  and  total 
wrong. 

Reliability:  No  data  available, 

306 


t 


S;i in | »K*  it «Mii :  Carnival : 

A.  Sin tighter. 

15.  Bn  nonet . 

(’.  Funeral. 

I  >.  Sen.-nal  | >«T"« >n . 

10.  Festival. 

Tot  :  (  'ompn.-.-  ( )rient  .it inn,  (Tt‘»tWiA. 

Drseript  inn  :  I  lie  .-object  :i>'iniu,>  that  lie  i-  in  ;i  1 1  air*  raft.  Me  is 
K»lil  in  what  direction  In*  i.-  llyino,  and  th  t  lie  make-  a  t urn.  rij_dit  nr 
left.  Ills  t iislv  is  tn  determine  lii>  new  direction.  Tin*  te-t  is  hi'dily 
-peeded  si  Hit  5  minutes  tin*  allowed  tn  dn  tin*  150  items. 

Number  of  items,  150. 

'I’inie  limit.  5  minutes. 

Scoring  formula:  'bun  scores  were  obtained,  total  ri^lit  and  total 


w  ron<jp 


New  direct  ion 
Wed  ( jitisWer). 


Reliability :  No  data  available. 

Sample  items :  Von  sire  living —  and  turn 

1.  North  left 

'1.  West  ri<:lit 

.").  North  rifilit 

'best  :  E-timation  of  length,  CP<»3lA. 

Description:  Five  bats  of  standard  lengths  arranged  in  order  from 
A  to  10  are  shown.  Part  I  of  tin1  test  consist-  of  bar-  of  diiTerent 
lengths,  'bhe  task  i.-  to  match  each  bar  with  the  correct  standard  bar. 
In  part  II  of  the  test  the  satin*  standard  bar.-  are  n-ed.  'bhe  item  bar-, 
however,  are  double  length,  'bhe  task  i-  to  determine  which  -tandard 
bar  lnts  been  doubled  to  form  etieli  item  bar. 

Number  of  items: 

Part  r  :  75. 

Part  11:75. 

l  ime  limit : 

Part  1 :  1  minutes. 

Part  II :  5  minutes. 

Seorinii  formula  :  Two  .-cores  were  obtained,  total  rijiht  and  total 


wrono. 

Reliability:  r  « M ».'»  for  rights.  n.7-_'  for  wrongs.  Correlation  !*•- 
iween  part  I  and  part  II  corrected  for  length.  X 


f) 
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Ansyre  r ; 


S.implc  it i*iii  : 

/ 


C-L."plo  Problem 


A  3 


STANDARDS 


Fit. no:  A.'i.'i. — K>l'mmlion  of  I.cninh  Test,  iTl'dlA. 


Tliis  s ;  1 1 1 1 1  <  I  *  *  Imr  is  <ti-n  to  li<'  of  Hit-  same  lni^lli  ;is  si  :t  i  k);i  rd  It.  'Flu*  correct 
;i iiswor  to  i li is  item,  t lien,  is  1!.  fonscqucntly  tin*  space  tnnlcr  15  In  Hit*  sample 
answer  Inis  I  icon  blackened. 


Test :  Flight  Orientation,  CP5*2SA. 

Dost  ription :  In  tliis  test  earh  item  eoiisists  of  two  pictures.  The 
j>iet me  at  the  left  shows  a  cockpit  view  of  the  terrain  over  which  nil 
aircraft  is  living.  The  picture  at  the  right  shows  the  same  cockpit 
view  as  it  appears  after  the  plane  lias  performed  a  single  maneuver. 
The  task  is  to  determine  which  one  of  six  possible  maneuvers  the  plane 
has  performed  :  left  or  right  turn,  left  or  right  roll,  climb  up  or  down. 

Number  of  items: 

Part  I:  17. 

Part  II:  50. 

l  ime  limit : 

Part  I  :  S  minutes. 

Part  II :  11  minutes. 

Scoring  formula:  Two  scores  were  obtained,  total  right  and  total 
wrong. 

luT.ability  :  r  0.7s.  Correlation  between  part  I  and  part  II  cor¬ 
rected  for  length.  X  =  502. 

Test:  Mechanical  Information,  CI1KI5B. 

Description:  This  test  contains  verbal  items  concerning  automobile 
mechanics  and  the  use  of  tools. 

Number  of  Items  :  00. 

Time  Limit  :  1*J  minutes. 

Scoring  Formula:  Two  scores  were  obtained,  total  right  and  total 


wrong. 
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|,n"k  ill  S;i  u;  i  >l«  •  1  ’  !*■  >1  •  1 1  •  1 1 1  I  In-low.  Two  |  >  h  •  1 1 1  r>  — ;  ;nc  -lioun.  Tin-  )•><-!  1 1  n  ■  ;ii  :  I  • 

■  I!  .-h.'U  s  :i  kpii  \  i  i  •  w  .  Tin-  i  •ii-i  1 1 1  <  •  m  tin-  i  i  _•  li  1  iIh-  i .  •«•*  pit  \  i<-\\  :is 

:i |i|!i';i i  '  ;i  f 1 1  r  ;i  sinul<'  uui m-m  t-r.  Your  I : i -k  i-  •I.  iri  iiiiiic  1 1 1 <  iiiam  in  or. 
In'  inn iii-iiv  i-r  will  1  m *  uni'  uf  Hu-  V i •  1 1 . > w i 1 1 u  ;  1  .* -f i  nr  Ki-^lii  Turn,  l.i'fi  nr  llijlii 
,"11.  Cliihli  l ' 1 1  nr  I >i i w 1 1 . 

Wlml  k  1  In*  1 1 1 : 1 1 1 ■  ■  1 1 \  •  r  in  S;i ui| ill*  1? 

•iiiLu',—  riT^^^fFjiirsnBirTrri 

{ 


Fk.i  uk  A.'Jl. — 1'li^lil  <  >r  ii-ni :i l  inn  T*  >t  (T.TJS.\. 


Hull  ri;:lil  is  roi,n,e|.  Tin'  lirsl  |ii»nirc  shows  llic  rock  pi  i  \|ow  ns  li  appears 

w 1 1 i l •  •  living  si r.ii_'lil  niul  level  liiwnri!  n  1  :i i 1 1  rain:o.  Th<*  smiml  picture 

shows  ill**  view  :is  ;i  : i ) >) o •; i rs  in  n  riulil  hank.  To  mow*  from  ) In*  lirsl  po^himi  io 
lln*  mvoihI  tin*  pili >1  Inis  lollnl  1 1  h*  ] *1  ;■  im*  in  1 1 1«*  rijjlit.  TIm*  answer  in  hem  Nn.  1 
sin  mill  In1  niiirknl  ns  shown  oil  vour  nnswr  slice). 


Ueliabilit y  :  Xo  data  available. 

Sample  Item:  A  main  bearing supports  a — 

A.  camshaft. 

B.  universal. 

(\  driveslmft. 

I).  erankslia ft. 

Test  ;  Memory  for  Landmarks,  Ci.MoA  X'2. 

Deseript ion  :  A  page  containing  1*»  lamlmarks  (e.  g.  rivers,  lakes, 
etc.) .  eaeli  uf  which  is  named,  is  studied  fur  I  minutes.  The  page  is 
then  turned  and  the  same  landmarks  are  shown  without  the  names. 
The  task  is  to  select,  from  a  list  of  l.*»  names,  the  eorreet  name  for  each 
landmark. 

XumhiT  uf  Items :  Three  parts  with  l‘J  items  per  jiart. 

'Time  L  uiit  :  I  minutes  study  period  per  part,  and  1  minutes  to 
answer  li!  items  per  part. 

Scoring  Formula:  'Two  scores  were  obtained,  total  right  and  total 
wrong. 

Ueliahility :  r  -H.Mi.  Correlat  ion  hetweeii  part  I  and  part  II  of 
form  ('IoK)AX  1  corrected  for  length.  X~*2-hS. 

Sample  Items:  Landmarks  to  be  studied. 


309 


ROBIN  CREEK 


SPARROW  CREEK 


CANARY  CREEK 


I - - - ! - - - ! - J 

Fii.ua:  A.  27.  Mnnury  fur  l:imlmark«  lot  CI.M0AX2  ( samples) 

Two  sample  problems. 


A  ROBIN  CREEK 

B.  SPARROW  CREEK 

C.  CANARY  CREEK 

Fii.uik  A. 215.--  Memory  fur  landmarks  1  »*s  1  (.‘I710AX2  (problems) 


Te.-t :  Numerical  Operation,  CI702BX1. 

Don  option  :  This  tot  contains  <>7  problems  in  addition,  nmltiplica- 
tion.  siihl ractioii,  and  division.  Whole  numbers,  fractions,  decimals, 
and  percentages  are  used.  Ten  items  ret  pi  ire  approximate  answers  to 
more  complicated  problems. 

Number  of  Items :  7a. 

'rime  Limit  :  10  minutes. 

Scoring  Formula:  Two  scores  were  obtained,  total  ri^bt  and  total 
wroiij;. 

Reliability:  Correlation  between  the  fust  and  second  half  of  form 
C 1  7o_> I {  is  n.iis.  'I  bis  is  the  lower  limit  of  the  reliability  coellicient  as 
the  i  wo  parts  are  not  comparable  in  content.  X  1774. 

Sample  Item:  Perform  the  following  numerical  computations: 

Add :  7  ;  S  111  (A)  .71  (IS)  77  ((’)  7(5  (D)  77  (K)  78. 

(A)  10S.SJ). 

0,127X8 

Approximate:  (C)  141.2. 

(l>)  204.1. 

(K)  8 12.2. 
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Test  :  Patton  ( ’<  hi  1 1  ncl  icii  'it  *1  ( ,  (  ‘  1  A. 

I  )i-— -ri I >1  it iii  :  This  t t*:-i  ret  pi  ire-  the  ~ul *jfi  t  in  vi-uali/c  tin-  rein t ion  - 
- ' i i | »  between  a  pattern  drauinc  ami  the  object  it  represent-.  Kadi 
i  fin  consists  of  two  tlra \viu^>.  The  drawing  on  the  left  rep: v-c ut-  a 
i  tire  dime  n-ional  object.  On  tin-  ri^ln  i-  a  pattern  drawin«r  nf  the 
.  Iijci't.  '1  lie  cdces  (if  (Ilf  nlijfft  a IV  liniiil icrctl ;  the  et]ccs  tm  tin-  pattern 
.  iv  lettered.  1  lie  task  is  tn  matt  h  each  muuhrrrd  ct|ce  with  a  lettered 
,  i |f e.  Twt!  etl^o  ill  both  drawings .  lalieletl  X  and  0  are  always  cj\cii 
1 1 ir  re ferenec  purposes. 

Nninlier  of  Items :  :>0. 

Time  Limit  :  la  minutes. 

Scoring;  I'oruuila:  1  wo  .-cores  were  nlitamed.  tutal  ri^lit  and  total 
wrong. 

Reliability:  No  data  available. 

Sample  Items: 


I  rones  ponds  to  (A)  li  (H)  p  ((’)  f  (I))  t  (K)  k 
•J  corresponds  to  (A)  t  (II)  f  ((’)  li  (I))  k  (K)  p 
'best:  Pattern  Identification,  CPS’JOA. 

Description:  Kadi  item  consists  of  two  croups  of  small  circles. 
The  croup  on  the  left  consists  of  four  circles  which  form  a  definite 
pat  tern  by  reason  of  their  size  ami  rein  I  imiship  to  each  other.  In  this 
croup  there  is  also  a  small  cross.  On  the  right,  in  a  larger  circular 
area,  the  pattern  found  on  the  left  is  repeated.  There  are,  however, 
additional  circles  which  make  more  complex  the  identification  of  the 
pattern.  In  addition,  the  basic  pattern  of  four  circles  is  at  tunes 
placed  to  one  side  of  the  area  so  that  a  part  or  all  of  some  of  the  circles 
in  the  pattern  is  not  shown.  Five  lettered  crosses,  one  of  which 
corresponds  to  the  cross  in  the  pattern  on  the  left  are  shown  in  the 


7ti:t::.7  47 


.’111 


complex  pattern  on  tin;  light.  Tin*  subject  indicates  Ids  choice  by 
clmo'ing  the  correct  lettered  cross. 

Number  of  Items : 

Part  1 :  24. 

Part  11  :  24. 

Time  Limit: 

Part  1  :  0  minutes. 

Part  11:7  minutes. 

Scoring  Formula :  Two  scores  were  obtained,  total  right  and  total 


wrong. 

Keliahility  :  r  0.81.  Correlation  between  part  I  and  part  II  cor¬ 
rected  for  length.  N--1G8.12 
Sample  item: 


Khu'hk  A.'JS.  Piitlern  Mentineatieii  Test,  (TS’JOA. 


For  this  sample  the  correct  circles  in  the  pattern  have  been  Idled  in. 
The  cro.-s  P>  is  t lie  correct  answer  for  this  problem. 


'Test  :  Polar  Grid  Coordinate,  CPS19B. 

1).  ■script ion  :  'Flic  subject  is  given  two  numbers  representing  the 
bearing  and  range  of  a  point,  from  the  center  of  a  circle.  lie  must 
plot  the  bearing  of  this  point  in  the  circle  which  is  calibrated  in  degrees 
azimuth,  and  must  plot  range  by  concentric  circles  within  it  indicating 
units  of  distance  from  the  center.  After  plotting  the  point,  he  must 
determine  its  coordinates  on  the  X  and  Y  scales  of  a  square  grid 
i  ircniiiscribing  the  circle. 

Number  of  items : 

Part  I,  20. 

Part  H,  21. 

'I'inie  limit: 

Part  I,  S  minutes. 

Part  1 1,  7  minutes. 


11  I’ronrehs  Iti'porl  for  Putlilliuli-r  Projrrt  of  AKKD  No.  1,  l  Srpt.  ISM  I,  p.  8. 
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JKLMWOAIODR  f  «  M  I  JKtMHOA/*COSrOHIJKLM«OA»CO 

Fiotm*  A.20.— Polnr  Grid  Ooorillnnto  Tca'i  OP8193.  msv-4 7  fr»»p.iir 


Srorino  formula.:  Twn  »n*-i  wen-  ohta ined,  total  li^lit  and  total 

w  roiio. 

Reliability:  r  <».!)().  Correlation  obtained  |»v  split -half  technique 
niiTirl ed  for  Idiot  1).  X  100.u 

Sanijilc  item:  ('uric  location  numbers.  177-1 X .  See  Figure  A 
A imsver  to  Problem :  K  .1. 

Te~t  :  l’o'it  ion  Oriental  ion.  CP.VJtiA. 

I  )e~eripl  ion  :  Parts  I  and  1 1  of  this  test  consist  of  drawings  of  hands. 
The  task  is  to  determine  whether  each  hand  is  a  left  hand  or  ri"ht 
liand.  Parts  III  and  1 V  contain  draw  inirs  of  hand",  anus,  feet,  le^s, 
and  eyes.  The  task  njzain  is  to  determine  whether  each  is  the  left  or 
riydit  lnemher.  A  special  answer  sheet  is  required  fur  this  test. 

X ntnher  of  items : 

Part  r,  ‘20. 

Part  11,00. 

Part  1 1 1,  20. 

Part  IV,  do. 

Time  limit : 

Parts  I  and  11,7  minutes  per  part. 

Parts  1 1 1  ami  I  V,  71,  '■>  minutes  per  part. 

Scoring  formula:  Two  scores  were  obtained,  total  ri^ht  and  total 

Wl'ime, 

Reliability:  /•  - O.S.'k  Correlation  between  part  T  and  part  11  cor¬ 
rect  ed  for  length.  X-.m 
Sample  Items: 


A 

r\ 

a 

l\  A 

c 

D 

C 

\L  vx  Cx 

m\) 

O  -  -  - 

uJJj 

-:*vV 

\J  / 

<J  .  -  | 

^  -  :*x 

A/ 

/ 

l*  7 

^  S' 

lip/ 

Fii.i  iu.  A l’lisiitmi  ( Irteiilatloii  Tr«t,  Cl’A-itA. 


Test:  Ratio  Estimation  C'P22a  A. 

1  Inscription  :  The  subject  is  presented  with  pairs  of  lines  of  different 
lengths.  The  task  is  to  determine  the  proportion  of  tin-  shorter  line 
to  the  longer  line. 

linSltD  Urport  No.  1st::.  Formal  Mnnor.wnlum  No.  I,  rr*  Uinlnar>-  ;-*rt  of 
from  <  Nfillii*- rtipi*  upiTalor 
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X umber  of  it «*nis  : 

I’m  t  I.  in. 

Pin  t  11.  in. 

Time  limit  : 

l’art  1.  1  minutes  15  seconds. 

Part  1 1 .  3  minutes  l.*»  sen  tin  Is. 

Scoriim  formula:  Two  scores  were  nlu.iined,  total  right  and  total 
wrong. 

IhTinbilit  v  :  /■  O.'Jti.  Correlation  obtained  In  split  half  technique 
eorreeteil  for  lengt  1>.  X  -  lt)().u 


Sample  items : 


12  3  4  5 

Fim  uk  A.:M  — Uall.1  Kslliiiitlti.il  Test, 


Test :  Seale  Heading,  CPG37A. 

Description:  DiUVrcnt  types  of  scales  of  varied  complexity  are 
presented.  The  siihjeet  must  read  one  or  more  points  on  each  scale. 

Number  of  items,  70. 

'rime  limit,  15  minutes. 

Scoring  formula:  Two  scores  were  obtained,  total  right  and  total 
wrong. 

Reliability:  r  0.S1.  Correlation  obtained  by  split  half  technique 
for  length.  X  *- 100.18 

Sample'  items : 


1  *  ( * S It J >  Nii.  1  s:t.  I’i.rmiil  Mrmunmilmii  Nn.  -I.  I'rrllinlimry  Kr|'"rt  »f  Itrsult* 

frnin  Om11Ih«i-h|ii*  OiM-riitur  Tfh(». 

/Mi/. 
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I  f  t  -  Spill  i : 1 1  K t • : i  - '  1 1 1 i  h f ,  (  I  'M  !  K  X ‘2 . 

I  )r  -f|  l  pi  Hill  :  I*  *  1 1*  i  * .  i  ■  *  1 1  1 1  fill.  ;i  p.'  I  1  l>  ll  lit  I  I  II  i  Id  rill  1 1 1  if'  I  lie  pc  i'll  H  >11 
if  \  II  ill'll'-  ill  ii  '  ene-  "  f  c  I  i  i '  1  i  * 1 2 '  ill  III  f  ;l  p'.  In  m  III  I.1  ill'!  mil  !•'  :  lit*  rule 

_•>  |\ (Til  1 1 1  pr  1 1 II'  p  1 1 1 1  1 1 1 1 1  (if  till'  'Vlllli'il'  i'  llil'iil  dll  llicil'  |n,:u  inn  ill 
flat  it'll  III  Lr;i|f.  olli'T"  ill  relation  III  till'  d.i'lii'.  .Mill  'lill  i  >1 1 1 1  •  I  ‘  ill 
irl, limn  Id  lull  1 1  ”:ipi  iiiicl  ilii'lii".  I  Iik  in  k  i'  In  di'cnier  an*1  applv 
i  lie  in  If  fur  each  il»‘ii>. 

N umber  uf  items,  70. 

'lime  limit ,  *J.">  minutes. 

Scoring  f  i  nil  t  ilia  :  Tun  scorc>  were  obtained,  luinl  riijil  a  lit]  (dial 
win  11^. 

Iidiabilit  v  :  r  0.>d  fur  furiu  (  Tj 1 1  BX  I .  C  'ui  iflal  inn  1  u-l  w  »*ci i  pari 
1  ,iinl  pari  1 1  eon  ci  ted  for  length.  X  22  I.1*1 
Siiinph'  ili'ins : 


I  n  tlic  a  l>u  ve  mi  in  1  )1»*.  nut  in*  tlial  (In*  numerals  1  and  2  arc  l  mill  plan'd 
ju.'i  lu  tin*  riprlil  of  tlu*  <_mps  in  the  fust  four  rmv>.  'l  l u*  problem  is  l<> 
determine  wln'ic  l hr  miiniTals  would  occur  on  l In*  Iasi  row.  In  this 
rase.  tlu*  n  ill*  is:  “Place  lln*  numerals  j  1 1  "l  liflit  of  I  lu*  "Ups."  Tin* 

1 1 1 1 1 1 icra  1  1  would  finis  lu*  placed  on  tin*  1  i 1 1 1 ■  almvc  1)  and  lln*  numeral 

2  would  In'  plan'll  on  lln*  1  i m*  almvc  II,  1)  ami  II  an*  llicrcfun*  tin* 
c  urri'ct  aiKwrr.'  for  problems  1  and  2. 

Tr.-l  :  Spci'd  uf  I  dent  ilical  ion.  (T’GUtC. 

1  >i '■  riplioi; :  Ilcnis  arc  in  croups  of  four,  cadi  ”r«nip  coii"i'linf  of 
four  iiiiiiiIkti'i]  dc.'i f us  and  live  Icllcrcd  de'i^iis.  Four  of  lln*  leiiered 
designs  are  (lie  same  as  (lie  four  numliered  one',  The  link  is  lu  nuiteh 
r adi  lettered  design  with  the  idcnlie.il  numbered  design. 

N uinlicr  of  items,  ‘.Hi. 

Time  limit,  f>i.j  minutes. 

.Seornif  f 1 1 r i 1 1 1 1 1  a  :  Two  scores  wrre  obtained,  total  ri^lit  ami  lutui 


wdnir. 

K  liability:  r  0.70  for  form  Cl’illnA.  C'orrelal ion  between  sep¬ 
arately  limed  halves.  N--1,H!K).U 


:ur> 


I*  r.u-.lforil.  J.  r.  nnd  Ijirry.  J.  I.,  r</«.  op.  fit..  fli«plfr  7. 
"  Ihhl  .  ■lmpt.r  lrt. 


Sample  items: 


l'lo.  A.  31.— Speed  of  Ideiit  Meat  Ion  Test,  CT010C. 

The  correct  answers  are  1,  C;  2,  E;  3,  D;  4,  A. 

Test:  .Spot  Location,  CP818A. 

Description:  The  subject  is  presented  with  a  large  circle  which  is 
divided  into  lettered  areas.  Surrounding  the  large  circle  are  smaller 
circles  in  which  are  numbered  dots.  The  task  is  to  determine  in  which 
lettered  area  of  the  large  circle  the  dots  in  the  smaller  circles  would 
fall  if  the  smaller  circles  were  the  same  size  as  the  large  circle. 

Number  of  items: 

Part  I,  10. 

Part  II,  -10. 

Tune  limit: 

Part  I,  •!  minutes  15  seconds. 

Part  II,  3  minutes  15  seconds. 

Scoring  formula:  Two  scores  were  obtained,  total  right  and  total 
wrong. 

Reliability:  r—  0.08.  Correlation  obtained  by  split-lmlf  technique 
con  vet  ed  for  length.  N=100.1# 

"OS!tl>  ]ti*|>i>rt  No.  18.T,  l-'oriniil  Mcniorniiiliiin  No.  4.  I’rrllinliuiry  Kei>ort  of  Ilosiilts  from 
OKcUlomi>i><>  Oprrntor  ToHt*. 
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-  **-*8,' 


I*  f*  ■  -*l  »■! 

Sample  items: 


The  correct  answers  are:  1,  M;  *2,  N :  d,  I);  4,  P;  “>,  E. 

Test:  Visual  Memory,  dull  A. 

Description:  Each  of  the  live  parts  of  this  test  consists  of  a  largo 
aerial  photograph  which  is  studied  for  1  minute.  The  page  is  then 
turned  and  24  small  aerial  photographs  are  presented  to  the  subject. 
'Fhe  task  is  to  determine  which  of  the  small  photographs  are  repre¬ 
sented  in  the  large  photograph  and  which  are  not. 

Number  of  items:  Five  parts  of  24  items  per  part. 

Time  limit :  One  minute  study  period  per  part ;  2  minutes  to  answer 
24  items  per  part. 

Scoring  formula:  Two  scores  were  obtained,  total  right  and  total 
wrong. 

Reliability:  /*“0.S7.  Knder-Richardson  method.  N  —  C24.19 

Sample  item  :  Photograph  to  be  studied. 


>'  Guilford,  J.  P.  and  [.aery.  J.  I.,  rrf».  op.  fit.,  rhaplrr  11. 


Flui.'itK  A.M.— Visual  Memory  Test,  Cl  all A  (Study). 


Now  look  at  the  sample  Items  below.  Is  item  1  a  section  of  tlie  sample  plate? 
Iioinomher,  you  eaniiot  refer  lmek  to  the  previous  page. 

Item  1  Is  a  section  of  the  sample  plate,  so  hlaekea  space  A  opposite  number  1  on 
your  answer  sheet  now.  Item  2  shows  a  road,  but  it  Is  not  the  road  on  the 
photograph  Just  studied.  So  blacken  space  11  opposite  number  2  on  your  answer 
sheet. 

Two  s:i  in  pie  problems : 

>  — - -  ■  1  ■  i  -  —  ■  — i 


Test  :  Radar  Student  I n i< m  Blank. 

Descript  inn  :  Radar  observer  students  completed  tills  information 
blank  at  tin*  time  tbe  experimental  battery  was  administered.  Tbe 
blank  contains  ipiestions  covering  where  tbe  students  bad  been  admin¬ 
istered  tin*  Radar  ( )bserver  Select  ion  Bat  t  cry  and  tbe  Air -(Yew  Chi -si¬ 
lica  t  mn  Bat  t  ery.  'I  be  st  intents  also  select  ed  t  be  one  of  four  st  a  lenient  s 
that  best  described  tbe  strength  of  their  desire  to  take  radar  observer 
training. 

Psvelioniolor  Tests 

Test:  (beck  List  Dial  Setting  (Model  A)  (no  code  number). 

Description:  I  bis  test  consists  of  four  dials,  each  calibrated  in  dis¬ 
crete  steps  from  1  to  11.  The  subject  is  given  ji  ]i>t  of  settings  for  tbe 
four  dials.  lie  must  set  tbe  dials  according  to  tbe  list  and  throw  a 
toggle  switch.  I  f  tbe  settings  are  correct,  a  light  flashes  and  be  goes  on 
to  tbe  next  group  of  settings. 

Number  of  items,  2  trials. 

'I'iine  limit ,  f>  minutes  per  trial. 

Scoring  fonnidi:  Total  number  of  correct  settings. 

Reliability:  /*--u.70.  Odd-even  correlation  corrected  for  length. 

N- 381. 20 


I'lio  ia:  A..’V>.--('hci'k  List  Iilal  Selling  Test  (iiumIcI  A). 

Test:  Complex  Coordination,  CM7t)lK. 

Description:  This  is  tbe  same  as  tbe  model  u-ed  in  tbe  Air-Crew 
Classification  Battery  with  two  exceptions.-  First,  tbe  contacts  for 


Mi'llon.  A.  W..  ((/.  .1  iipuriilun  hul".  A.\K  nvli01«M  ps\i  liulo^y  (T.^rniu 
imrln,  no.  I.  WnxliliiKlon  :  iiovi-nmu-iu  I’rtnfluu  Ollli-o,  11M7. 


r»--  --.i r*-li  iv- 
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tlu*  lights  arc  iniK'li  sni:i Her.  Tims,  more  exact  movement  of  the  con 
trol  is  necessary  to  keep  the  lights  on.  This  calls  for  liner  motor 
adjustments  on  the  part  of  the  subject.  Recoil*!,  the  space  between 
contacts  is  correspondingly  larger,  Tims.  the  time  between  one 
light  going  olf  and  the  next  one  coming  on  is.  greater. 

Number  of  items:  One  test  period. 

'Fime  limit,  8  minutes. 

Scoring  formula  :  Total  number  of  correct  settings. 

Reliability:  /,n=0..S5.  Odd-even  correlation  corrected  fo  length. 
N  =  .‘581. 21 

A]>parntus : 


Fiona-:  A.U. — Complex  eoonlliintion  test,  see  MI01A. 

Test :  Rate  control,  CM825A. 

Deseript  ion :  The  task  is  to  keep  a  pointer  on  a  black  line  which 
moves  hack  and  forth  across  a  curved  window  at  a  varying  rate  of 
speed.  The  rate  of  movement  of  the  pointer  is  controlled  by  turning  a 
knob  to  the  right  to  move  the  pointer  to  the  right,  and  turning  it  to 
the  left  to  move  the  pointer  to  the  left.  The  further  the  knob  is  turned 
in  either  direction,  the  faster  the  pointer  moves  in  that  direction. 

Time  limit,  8-minute  trial. 

Scoring  formula:  Total  time  pointer  is  kept  on  black  line. 

»<  Ibid. 
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Reliability :  r  d.Sl.  Odd-even  correlation  corrected  for  length. 

X  -=:>S1."2 
Apparatus : 

» 

t 


i-'io.  A.."'.).— Hale  Colli  rot  Test,  C.MVJ.IA. 

Test:  Self-Pacing  Discrimination  Reaction  Time  ((TfdlK  modi¬ 
fied). 

Description:  This  test  is  much  the  same  as  the  corresponding  test 
in  the  Air-Crew  Classification  Battery  with  two  main  differences. 
First,  instead  of  lights  of  different  color,  this  test  uses  lights  of  differ¬ 
ent  intensity.  The  relationship  of  the  dimmer  to  the  brighter  lights 
determines  which  switch  to  throw.  Second,  instead  of  the  pattern 
of  the  lights  changing  periodically  and  the  score  being  the  time  re¬ 
quired  by  the  subject  to  throw  the  correct  switch  for  each  pattern, 
this  test  is  self-paced  and  the  pattern  changes  only  when  the  subject 
throws  the  correct  switch.  When  he  makes  an  error,  the  lights  go 
oir  and  he  must  wait  for  the  pattern  to  be.  repeated. 

'rime  limit,  S-miunte  t rial. 

Scoring  formula  :  Total  number  of  correct  responses. 

Reliability:  /•— ft.'.H).  Odd-even  correlation  corrected  for  length. 

X  =  381.2* 


*»  ri>ui. 

»  thirl. 
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A  pparatus : 


Fit:.  A.  III.  Self- -I'licing  lMscihnluailou  Head  loti  Tluu*  Tcsl ,  (Till  IF.  (modi  lied). 


Test :  Two  -Hand  Pursuit  (Thurstone.) ,  CMS10A. 

Description :  This  test  is  siiuiliar  to  the  Two-Hand  Coordination 
Test  in  the  Air-Crew  Classification  Ihitterv.  In  this  form,  the  lathe 
controls  move  a  surface  which  is  rotating  eccentrically.  The  task  is 
to  keep  a  contact  spot  on  the  moving  table  under  a  stationary  contact 
button. 
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Time  limii ,  S-minnte  ( rial. 

Srnriii^r  formula:  total  I  inn*  that  contact  ."put  on  moving  t  ;i  1  >  lo  is 
kept  miller  stationary  contact  Imttoii. 

Reliability :  r  —  d.^i.  Odd-even  correlation  corrected  for  length. 
N  =  :>si.-‘ 

Apparatus: 


t 


/ 


Fin.  A. II. — Two-Hand  I'ursnil  Tesi,  OIM0A  iThursloiic). 

'l  est  :  Visual  Coincidence  Reaction  Test  (CRtlloB.'*  modified). 

Inscription:  Two  stationary  lines  of  li<;ht  on  the  same  horizontal 
plane  appear  at  a  window.  A  moment  later  a  moving  line  of  lijjht 
appears  at  the  top  of  the  window  and  moves  down  between  the  station¬ 
ary  lights.  The  task  is  to  throw  a  toggle  switch  at  the  moment  the 
movin';  li/dit  is  in  line,  with  the  stationary  lines.  In  successive  trials 
iho  stationary  lines  appear  at  diirerent  places  alonj;  the  window  and 
the  inoviim  line  travels  at  diirerent  rates. 

n 

Number  of  items,  loti  trials. 

Time  limit,  none. 

Scoring  fornmla:  Total  nnmher  of  correct  responses. 

Reliability;  /•==0.7o.  Oild-even  correlation  corrected  for  length. 


»  IbUi. 
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Apparatus: 


i 

1 


\ 

l 


Kl<*.  A  ll', — Visual  Coinridcnrr  Itruct  init  'IVsl,  (TdllJIW  ( mod IHimI ). 

Glossary  of  Technical  Terms 

Ansm.i  rB  Ai/m ri)K. — Height  of  an  aircraft  above  the  terrain. 

AFC’. — Automatic  frequency  control.  The  circuit  or  its  control 
which,  once  properly  adjusted,  maintains  the  radar  receiver  in 
tune  with  the  transmitter. 

Ai minis  Point. — The  landmark  used  as  a  reference  for  establishing  the 
bombing  run.  It  is  usually  near  the  center  of  the  target  area,  ex¬ 
cept  for  olf-set  bombing. 

Am  Purr. — A  navigation  procedure  for  determining  and  recording  air 
position. 

Am  Poor  Wind. — The  wind  direction  and  velocity  in  terms  of  hourly 
units,  obtained  by  comparing  ground  position  and  air  position. 

Am  Position. — Also  called  no  wind  position.  A  theoretical  position 
of  the  aircra  ft  computed  from  true  air  speed  and  heading  only,  as¬ 
suming  no  wind  effect.  It  coincides  with  ground  position  wlmn 
there  is  no  wind. 

Am  Si’v.Ki), — The  velocity  of  an  aircraft  with  reference  only  to  the  air 
through  which  it  is  flying  and  without  reference  to  the  ground. 

Ai.tithik  Dinar. — The  electronic  time  delay,  or  its  control,  which 
eliminate*  the  alt  itude  hole  or  blank  area  in  the  center  of  the  scope, 
thus  reducing  distortion.  The  altitude  hole  results  from  the  ab¬ 
sence  of  relied ing  objects  between  the  aircraft  and  the  earth. 
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AX/APQ.  AX  APS. — Designation  for  air  1  •unit*  radar  equipment, 
'llir  following  arc  the  *-ets  referred  to  in  the  report. 

AX/APQ-5. — Kssent  ially  an  electronic  bomhsight,  n-rd  in  coil  junc¬ 
tion  with  a  radar  search  set. 

AX/APQ-7. — A  radar  search  set  known  as  the  Eagle.  characterized  by 
unusually  high  definition:  particularly  successful  as  ar.  aid  to 
bombing.  It  does  not  have  a  .‘>fi<P  sweep,  the  area  pre>entcd  on  the 
scope  being  limited  to  :»tP  to  the  left  and  :»<»'’  to  the  right  of  the 
heading  of  the  aircraft.  The  antenna  is  fixed.  The  set  is  with¬ 
out  azimuth  stabilization. 

AX/APQ-13. — A  radar  search  set  used  as  an  aid  to  both  bombing  and 
navigation.  It  lias  a  3(‘>0°  sweep  and  is  equipped  with  azimuth 
stabilization.  Antenna  tilt  control  allows  for  improved  scope 
presentation.  This  set  was  used  almost  exclusively  in  the  Pacific 
theater. 

AX/APQ-23. — The  most  improved  radar  search  set  in  production  at 
the  close  of  the  war,  characterized  by  simplified  and  automatic 
controls  and  computing  devices,  including  the  electronic  solution 
of  olFset  bombing  problems. 

AX/A  PS-15- A  radar  search  set  similar  to  the  AX/APQ- PI,  hut 
equipped  with  an  A -scope  to  facilitate  tuning  and  calibration.  It 
lias  fewer  separate  units  than  tin*  AN/A PQ -Id  and  therefore, 
defective  parts  are  not  as  easily  repaired  or  replaced.  This  set 
was  used  almost  exclusively  in  the  European  theater. 

AN/APS-15A. — An  improved  model  of  the  AX,  A  PS-15  set,  having 
a  different  computer  and  different  procedures  for  adjusting  range 
marks  and  for  calibrating  altitude  and  range. 

A-Sc’orn. — A  cathode  ray  tube  on  which  range  data  are  proented  as 
vertical  pips  on  a  horizontal  scale  of  distance.  It  is  used  to 
facilitate  tuning  and  calibration  on  the  AX/APQ-7,  the 
AX/APS-15,  ami  AX  APS  15 A  sets.  On  the  AX  APQ-7  it  is 
also  used  to  facilitate  setting  in  absolute  altitude. 

Automatic  Pmn*. — Also  called  (’-1.  because  of  the  popularity 
of  that  particular  model,  and  A  FOE,  automatic  flight  control 
equipment.  A  gyro-stabilized  electrical-mechanical  device,  for 
maintaining  the  aircraft  at  a  de.-ired  altitude,  l.’sed  cxteiisivclv  in 
all  theaters  and  in  training,  particularly  during  bombing  runs 
and  formation  flying. 

Azimuiti. — Angular  di.-tance  measured  in  degrees  clock wi-e  from 
true  north. 

Azimuth  St.uwi.ization'. — An  electronic  device  incorporated  in  the 
radar  set,  maintaining  north  at  the  top  of  the  ^copc  to  facilitate 
scope-map  interpretation.  It,  is  operated  from  the  flux-gate 
compass. 
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Bkacon. — Also  radar  beacon  or  Karon.  A  li.\<*<|  ground  radar  sig¬ 
nal  generator  which.  upon  being  activated  by  certain,  radar  sig¬ 
nals.  transmits  a  coded  signal  that  can  be  identified  on  the  air 
borne  scope  and  plotted  as  any  other  return.  Used  particnlarb 
for  homing. 

Bi.n\ — See  Pip. 

Bo. mii  Ki  — The  final  approach  to  the  target  during  which  final  cor- 
corrections  an*  made  in  course  and  in  comput  ing  the  release  point. 

Bo.Mn.siniiT. --Usually  refers  to  the  Xenlcn  sight.  An  optical  com¬ 
puting  imt rnment  used  in  visual  bombing  to  determine  ami 
direct  the  aircraft  to  tin*  bomb  release  point.  Its  solution  of  the 
bombing  problem  involves  the  handling  of  such  factors  as  true 
air  speed,  absolute  altitude,  and  drift. 

BTC). — Bombing  through  overcast.  An  Army  designation  for  air¬ 
borne  radar  operation.  See  RO  (B). 

U.M.nut.NTio.N. — 'Hie  systematic  adjustment  of  the  receiver-indicator, 
the  range  unit,  and  tin*  computer  so  that  accurate  distances 
are  indicated  for  altitude  and  slant  range.  The  set  is  calibrated 
on  the  ground  by  a  mechanic  and  checked  in  the  air  by  the  radar 
observer. 

C.\.MKit.\  Bo.miuno. — A  simulated  bombing  run  in  which,  instead 
of  bombs  being  released,  the  accuracy  of  release  is  measured  photo¬ 
graphically.  At  least  two  pictures  are  taken  of  the  ground,  one 
at  ‘‘Bombs  away”  ami  one  at  the  theoretical  time  of  impact.  Ac¬ 
curacy  of  measurement  varies  up  to  roughly  1)00  feet. 

('.vnioni:  Ray  Turn:  (CRT).' — Also  called  the  scope.  A  vacuum  tube 
in  which  an  electron  beam  is  made  visible  by  being  focused 
upon  a  fluorescent  screen  on  the  flat  end  of  the  tube  where  it  is 
converted  into  light  energy.  See  also  Radar. 

Cm.cK  Point. — A  landmark  identifiable  either  visually  or  on  the 
radar  seojx*. 

CntiTi.Ait  ICuicoit  (CE). — The  distance  between  the  point  of  impact  of 
a  bomb  ami  the  center  of  the  target. 

CooKiuNATi'.i)  Bo.miuno. — A  bombing  procedure  in  which  the  radar 
observer  furnishes  data,  particularly  speed  of  closure  in  the 
form  of  sighting  angles,  to  the  bombardier  who  can  then  set 
up  and  operate  bis  sight  mid  release  the  bombs  without  seeing  the 
target. 

Dr..\i)  Ri'.ckon i.nu  Xamoation  (I)R). — Inferring  the  position  of  the 
aircraft  bv  applying  to  the  last,  known  position  the  estimated 
track  made  good,  computed  from  a  previously  determined  wind, 
heading,  and  true  air  speed. 

Pm.ia  tion  Kicinm. — The  distance  the  bomb  falls  right  or  left  of  a 
line  extended  through  the  center  of  the  target  parallel  to  the  air¬ 
craft's  track. 
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Direct  Bombing. — The  bombing  procedure,  in  which  the  rathir  ob¬ 
server  directs  tiro  run  and  releases  the  bombs  independent  of 
the  bombardier.  Considered  to  be  less  accurate  than  coordinated 
bombing. 


Divinr.iis. — The  familiar  compass-like  instrument  used  to  measure 
and  plot  distances  on  maps  and  charts. 

Drift. — The  annular  dilFcrcnce  between  the  aircraft's  heading,  or 
direction  in  which  it  is  pointed,  and  its  track,  or  the  path  it  makes 
over  the  earth’s  surface.  Drift  is  zero  when  the  aircraft  is  headed 


directly  into  or  away  from  the  wind,  and  is  maximum  when  tho 
wind  direction  is  perpendicular  to  the  aircraft's  heading. 

Drietmkter. — Also  culled  the  B-3.  A  simple  optical  device  for  read¬ 
ing  drift. 

ETA. — Estimated  time  of  arrival. 

E-GB  Computer. — A  combination  circular  slide  rule  and  transparent- 
slide  vector  plotter.  It  is  used  to  make  conversions,  as  from  statute 
miles  to  nautical  miles,  and  to  compute  such  variables  as  air 
speed,  ground  speed,  wind  velocity,  wind  force,  and  ETA. 

Final  Point. — In  bombing,  the  last  sighting  angle  given  by  the  radar 
operator  to  the  bombardier  who  is  synchronizing  the  bombsight. 
After  the  final  point,  the  radar  observer  may  set  up  the  set  for 
direct  bombing,  in  the  event  that  the  bombardier  is  unablo  to 
make  the  release. 

Fix.— The  location  of  an  aircraft  from  terrain  features  appearing 
both  on  the  scope  and  on  n  map.  A  fix  may  consist  of  simultaneous 
bearings  on  two  or  more  features,  or  the  bearing  and  range  of  a 
single  feature. 

Gain. — See  Receiver  Gain  and  Video  Gain. 

Ground  Position. — The  point  on  the  ground  over  which  the  aircraft 
is  at  a  particular  moment.  May  be  expressed  ns  coordinates  of 
latitude  and  longitude. 

Ground  Range. — Distance  from  tho  ground  directly  under  tho  air¬ 
craft  to  the  object. 

Ground  Speed. — The  velocity  of  tho  aircraft  with  reference  to  tho 
earth’s  surface. 

IIAB. — Iligh-altitude  bombing.  See  RO  (B). 

Heading. — The  direction  in  which  tho  aircraft  is  pointed.  True 
heading,  in  respect  to  true  north,  is  obtained  from  a  compass  and 
deviation. 

H2S.— SeeRO  (B). 

II2X. — See  RO  (B). 

Initial  Point. — The  point  on  tho  ground  over  which  the  bombing 
rim  is  started. 

Knot  (K). — Nautical  miles  per  hour.  One  nautical  mile  equals  G,0S0 
feet. 
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LAB. — Low-altitude  bombing. 

Log. — In  navigation,  the  systematic  chronological  record  of  a;i  air¬ 
craft’s  flight. 

Loran. — Long-range  aid  to  navigation.  Two  ground  stations  trans¬ 
mit  synchronized  radio  signals.  The  air-borne  Loran  set  measures 
the  time  difference  in  receiving  signals.  Fixes  are  taken  by 
plotting  equipment  readings  on  specially  prepared  charts. 

Lummni.iXK. — An  illuminated  radius  of  the  scope  indicating  the  head¬ 
ings  of  the  aircraft.  It  points  to  the  top  of  the  scope  if  azimuth 
stabilization  is  ofT  and  to  true  heading  if  azimuth  stabilization 
is  on. 

Main  Soon:.— See  PPL  The  PPI  scope  located  on  the  radar  set. 

Mutko  Wind. — The  forecasted  wind  as  reported  in  briefing  and  used 
in  the  preflight  planning  of  the  mission. 

Mickey. — See  RO  (B). 

Mult  idle  Drift  Procedure. — In  radar  bombing,  a  systematic  method 
for  estimating  the  correction  in  heading  to  compensate  for  drift 
on  the  bombing  run. 

Offset  Bombing. — A  bombing  procedure  in  which  the  release  point 
is  computed  with  reference  to  an  aiming  point  outside  of  the 
target  area  in  order  to  bomb  a  target  which  is  not  visible  or  is 
poorly  visible. 

Pilotage. — Locating  and  navigating  the  aircraft  by  constant  refer¬ 
ence  to  the  ground,  either  directly  or  as  represented  in  the  scope. 
Contrasted  to  DR  navigation,  in  which  the  terrain  is  not  constantly 
observed. 

Pn\ — Also  called  blip.  The  presentation  on  the  scope  of  a  relative 
increase  in  current.  On  the  PPI  scope,  it  is  a  point  of  increased 
illumination  on  the  sweep  duo  to  a  target,  a  range  mark,  or  the 
bomb  release  mark.  On  the  A-scope,  it  is  a  sharp  vertical  peak. 

Precision  Turn. — Also  procedure  turn.  The  controlled  turn  of  an 
aircraft  or  of  a  formation  at  the  predetermined  rate,  usually  45 
degrees  or  more  per  minute. 

PPI. — Plan  position  indicator.  A  radar  cathode  ray  tube  on  the 
screen  on  which  the  terrain  under  the  aircraft  is  presented  by 
means  of  a  rotating  sweep  of  varying  brightness. 

Pulse. — A  momentarily  increased  current  or  voltage.  Radar  pulses 
are  timed  electronically  so  that  distances  are  known  from  the  time 
differences  between  the  transmitted  and  received  pulses. 

Radar. — Radio  detection  and  ranging.  Radar  includes  all  electronic 
pulse-echo  equipment.  An  air-borne  radar  search  set  is  the  basic 
unit  of  equipment  used  for  navigation  and  bombing.  It  transmits 
a  narrow  beam  of  high  frequency  radio  pulses  and  receives  the 
same  pulses,  in  weakened  form,  as  echoes  reflected  back  from  the 
earth’s  surface.  The  echoes  are  received  as  an  electron  beam  or 
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sweep  which  is  made  visible  l»y  being  focused  upon  a  fluorescent 
screen  on  the  flat  end  of  a  cathode  ray  tube  whore  it.  is  converted 
into  light  energy.  Different  terrain  features  reflect  different 
amounts  of  energy.  Water  reflects  nlmost  no  energy,  flat  country 
reflects  little,  while  cities  reflect  large  amounts  of  energy.  The 
electron  beam  constantly  sweeps  the  screen,  in  a  circular  motion 
for  search  sets,  of  which  the  AX/APS -15  is  an  example.  Tho 
result  is  an  illuminated  picture  on  the  screen  which  is,  in  effect, 
a  circular  map  of:  the  terrain  under  t he  aircraft.  The  point  on 
the  ground  directly  beneath  the  aircraft  is  the  center  of  the  map. 
Navigation  and  bombing  data  may  he  ascertained  accurately  with 
the  use  of  auxiliary  circuits  and  devices.  An  area  within  n  maxi¬ 
mum  radius  of  100  or  more  miles  is  represented,  regardless  of 
darkness  or  most  weather  conditions  such  as  undcrcnst. 

Radar  Beacon'.- -See  Beacon. 

Radar  Observer  (Bombardment). — See  RO(B). 

Range. — Distance,  measured  in  radar  by  the  time  required  for  a  pulso 
to  leave  the  transmitter,  be.  reflected  from  a  target,  and  return  ns 
an  echo  or  received  pulse. 

Range  Marks. — Calibrated  pips  on  the  PPI  sweep  which  present 
equidistant  concentric  circles  for  measuring  the  distance  from 
aircraft  to  target.  A  manual  control  places  range  marks  on  tho 
scope  at  either  1  or  5  nautical  miles  npait. 

Receiver  Gain. — The  manual  control  regulating  the  sensitivity  of  tho 
set  affecting  the  intensity  of  the  returns  only. 

RO(B). — Radar  Observer  (Bombardment).  The  Army  designation 
for  tho  occupational  specialty  0142:  commissioned  air-crew  mem¬ 
bers  subsequently  trained  as  radar  observers.  RO(B)  is  also 
used  to  refer  to  radar  observer  equipment,  training,  and  opera¬ 
tions.  RO(B),  a  more  recent  term,  is  loosely  synonymous  with 
BTO,  II2X,  IT2S,  HAS,  Mickey,  and  Stinky,  earlier  terms  which 
arose  largely  because  all  aspects  of  radnr,  including  the  term 
radar,  were  classified  as  secret. 

Score. — See  Cathode  Ray  Tube  (CRT),  and  Radnr. 

SCR-584. — An  electronic  device  for  measuring,  from  the  ground,  tho 
accuracy  of  a  simulated  bomb  release. 

SCR-717-A. — An  air-borno  radar  search  set,  now  ob-oletc.  Target 
returns  are  presented  on  a  “B”  scope,  a  vertical  sweep  on  a  rectan¬ 
gular  screen.  Resolution  is  poor,  there  is  no  azimuth  stabilization, 
and  no  computer  to  allow  for  bombing.  Tho  set  was  used  in  :;ea 
search  and  ns  an  aid  to  navigation. 

SCR-717-B. — An  improved  model  of  the  SCR--717-A,  also  obsolete. 
Target  returns  arc  presented  on  a  FBI  scope.  Resolution  is  poor, 
there  is  no  azimuth  stabilization  and  no  computer  for  bombing. 
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SCR-71S. — A  very  accurate  electronic  absolute  altimeter  operating  on 
the  radar  piilse-echn  principle.  Operating  range  is  from  2 5  to 
•10,000  feet. 

Slant  Hanoi:. — 'l'lie  shortest  straight-line  distance  from  the  aircraft 
to  the  object.  For  distances  exceeding  about  18  miles,  slant  range 
is  usually  converted  to  ground  range.  For  longer  distances, 
slant  range  and  ground  range  are  considered  equal. 

SiMNNKH. — The  radar  antenna,  so  called  because  it  constantly  rotates. 
The  rate  is  about  “2G  revolutions  per  minute.  The  antennae  of 
radar  search  sets  mentioned  in  this  volume  arc  of  the  spinner  type 
except  the  AN/APQ-7  which  has  a  fixed  wind-shaped  antenna. 

Stan  ink. — In  the  AAF  psychology  program,  a  composite  aptitude 
score  ranging  from  1,  low,  to  9,  high.  It  is  a  combination  of  the 
words  standard  and  nine.  Statistically,  it  is  a  function  of  the 
standard  deviation  of  combined  differentially  weighted  raw 
scores  on  a  battery  of  selection  tests.  Each  stanine  equals  one- 
half  of  the  sigma  of  the  distribution.  The  mean  and  the  median 
are  thus  a  stanine  of  5,  and  the  sigma  of  the  distribution  of  stanine 
scores  is  2  slanines.  The  percentage  distribution  of  stanines  is 
as  follows:  1,  4  percent ;  2,  7  percent;  3,  12  percent ;  4,  17  percent; 
5,  20  percent;  G,  17  percent;  7,  12  percent;  8,  7  percent;  9,  4 
percent. 

Stinkky. — See  RO(B). 

Swum*. — The  electronic  beam  or  its  motion  ns  it  moves  across  the 
screen  of  the  cathode  ray  tube.  On  the  PPI  scope  it  appears  as 
an  illuminated  radius  rotating  with  the  spinner  leaving  pips  on 
the  screen  indicating  terrain  features. 

Swkki*  Dki.ay. — A  device  or  its  control  for  extending  the  area  of 
terrain  represented  on  the  scope.  For  example,  with  20  miles 
of  sweep  delay  set  in,  the  area  represented  at  the  center  of  the 
scope  is  20  miles  from  the  aircraft. 

Tauokt. — The  point  or  area  to  be  bombed.  Loosely,  any  aspect  of  the 
terrain  giving  a  return  on  the  scope. 

Tauokt  Tim i no. — A  procedure  for  estimating  average  track  and 
ground  speed,  using  n  stop  watch  and  the  E-GB  computer.  Suc¬ 
cessive  fixes  on  a  target,  whether  or  not  it  is  identified,  nre  plotted 
on  1 1  io  transparent  slide  of  the  E-GB  computer  to  give  average 
track.  The  distancoand  time  interval  between  the  first  fix  and  the 
lust  fix  give  ground  speed.  The  procedure  lias  the  disadvantage 
of  requiring  the  target  to  remain  on  the  scope  for  about  nine 
minutes  mid  some  targets  move  off  the  scope  before  this  time.  The 
procedure  lias  tho  advantages  of  representing  an  average,  and  of 
allowing  truck  to  bo  estimated  without  an  exact  knowledge  of 
terrain. 
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Tilt. — The  control  which  adjn-ts  the  facing  of  the  antenna  to  tho 
ground,  bringing  up  desired  features  of  the  terrain. 

Tuackixo. — Adjusting  heading  so  that  the  aima  ft  parses  directly  over 
a  target.  Also  refers  to  following  a  target,  return  as  it  moves 
across  the  scope  face. 

Truk  1 1  kadi  no. — See  Heading. 

Turnino  Point. — A  radar  return  or  pair  of  coordinates  chosen  for  tho 
aircra  ft  or  formation  to  turn  upon.  It  may  he  at  any  point  on  tho 
mission,  for  example,  a  rendezvous  or  the  last  turn  before  tho 
initial  point  (IP). 

Video  Gain. — The  manual  control  regulating  the  intensity  of  all  trac¬ 
ings  on  the  scope,  including  ground  returns  but  not  affecting 
sensitivity.  Controls  the  general  level  of  illumination. 
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Abilities.  ( See  factors.) 
Administration.  (See  specific  test  or 
check.) 

directions  for.  (See  speclllc  test  or 
check.) 

test  administration,  64 
Administrative  control.  (See  profi¬ 
ciency  measurement.) 

Aerial  final  performance  check, 
adjusted  scores,  119 
administration,  118 
construction  of,  116 
description  of,  117,  146 
revisions,  120 
statistical  findings,  120 
types  of  Items,  117 

Aerial  Intermediate  performance  check. 
(Sec  also  aerial  final  performance 
check.) 

description  of,  1-16 
revisions,  120 
statistical  finding’s,  120 
Aerial  orientation,  CP520A: 
description  of,  303 
factor  content  of,  239 
Indicated  validity  of  factor  content, 
205 

(See  validation  of  experimental 
printed  test.) 

validity  for  radar-observer  training, 
269 

Airborne  oscilloscope  reading  (NDRC) : 

description  of,  212 
Air  Corps  vocabulary,  1912: 
description  of,  300 
factor  content  of,  20 
Indicated  validity  of  factor  content, 
207 

(See  validation  of  experimental 
printed  testa.) 

Air-crew  classification  battery.  ( Sec 
validation.) 

derivation  of  radar  otnnlne,  224 
description  of  teats,  2S3 
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Alr-erew  evaluation  and  research  de¬ 
tachment  No.  1.  (All HD  No.  1.) 
bombardier-navigator  comparison  by, 
222 

experimental  selection  battery  used 
by,  219 

Job  description  by,  217 
purpose  of,  210 
testa  constructed  by,  213 
validation  studies  by,  219 
Aptitudes.  (Sec  factors.) 

Area  visualization,  CP815A. 
description  of,  300 
factor  content  of,  20 
Army  general  classification  tost. 
Intercorrclntions  with  NDRC  proj* 
ect’a  tost,  214 

Air  to  surface  vessel.  (ASV) 
early  use  of,  0 

research  by  NDRC  project,  211 
Aviation  psychology  program. 

related  reports  from,  163 
Bench  Set  Final  Check, 
administration,  102 
apparatus,  09 
construction  of,  100 
description  of,  101 
revisions,  103 
statistical  findings,  103 
types  of  Items,  102 
Bench  set  trainer,  99 
Biographical  data  blank,  CEC02D. 
description  of,  2S3 

Indicated  validity  of  factor  content, 
253 

(See  validation  of  air-crew  classifier.- 
tlon  battery,) 

validity  for  radar-observer  training, 
203 

Bombardiers: 

miulrements  for  radnro  b  n  o  r  v  e  r 
school  sole'-  n,.  223 
belt  et Ion  for  radar  ob.  erver  training, 
224 

success  In  radar  rehool  compared  to 
navigators,  222 
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Bombardier  stanine.  ( S<v  v;i  llila  I  Inn 
of  nlrerew  specialty  stanlncs.) 
validity  for  radar  bombing  error.  I’d!) 
validity  for  radar  observer  training, 
2tii> 

I’omh  Souring.  Itl7 
mo'-alo  inotliod,  168 
notiinosalt'  method,  100 
photograph*  of  actual  bond*  drops,  108 
photographs  of  simulated  hmnb  drops, 
108 

recording  by  8CR-581,  gun  laying 
radar,  170 

Bombing  Proficiency  Scores,  171 
elmdar  error,  delhdtlon  of,  171 
deflection  error,  definition  of,  171 
range  error,  definition  of,  171 

Bombing,  (Si'C  Radar  Bombing.) 

Camera  bombing.  (.Sec  Bomb  Scoring. ) 

Cheek  list  dial  setting  (Model  A.). 
(.SYc  Validation  of  experimental 
psyelmmotor  tests.) 
description  of,  319 

Circular  error.  ( Sc/3  Bombing  profi¬ 
ciency  scores.) 

Combat,  rndar  observer  In,  47 

Compass  orlontntlon,  CICCOO. 
description  of,  300 
factor  content,  239 
( Sea  validation  of  experimental 
printed  tests.) 

Complex  coordination,  CM701A: 
correlation  with  success  In  rndar  ob¬ 
server  school,  219 
description  of,  299 
factor  content,  25,  239 
lidereorrelatlons,  220 
(.SYc  validation  of  air-crew  classifi¬ 
cation  battery.) 
weight  In  radar  stnnlnes,  220 

Complex  Coordination,  Model  0: 
description  of,  319 
(.Sen  validation  of  experimental  psy- 
ehomotor  battery.) 

Coordinated  bombing.  (See  rndar 
homldn  ) 

Coordinate  plotting  (NDRC)  : 
description  of,  213 
factor  loadings,  215 
Intercorrclntlona  with  NDRC  project’s 
tests,  215 


Coordinate  reading,  ('P22II5: 

correlation  with  success  In  radar  ob- 
.-er\er  school,  219 
description  of,  :">7,  213 
factor  loading*.  215 
liilereni  ridat Ions.  220 
(Srr  validation  of  radar  observer  se¬ 
lect  Ion  battery. ) 

validity  for  radar  observer  training, 
209 

weight  In  rnd..r  stanine,  224 
Course  grade.  (See  < trades  in  course.) 
Course  locution  test, 
description  of,  2!3 
factor  loadings,  215 

Dclh  cti'ni  error.  (*S’cc  Bombing  pro¬ 
ficiency  scores.) 

Dial  and  table  reading.  CP022A,  CPG21A. 
correlation  with  success  .ti  rndar  ob¬ 
server  school,  219 
description  of,  284 

Indicated  validity  of  factor  content, 
201 

( See  Validation  of  alr-erew  classifica¬ 
tion  battery.) 

validity  for  radar  bombing  error,  268, 
269 

validity  for  radar  observer  training, 
208,  209 

weight  In  rndar  stanine,  220 
Direct  bombing.  (£>Ye  Radar  bombing.) 
Discrimination  Reaction  Time,  CPG11D. 
correlation  with  success  In  rndar  ob¬ 
server  school,  219 
description  of,  292 
factor  content,  25 

(Sec-  validation  of  nlr-crew  classifica¬ 
tion  buttery.) 

validity  for  radar  observer  training, 
208,  269 
Drift 

correction  In  rndar  bombing,  43 
Estimation  of  length,  CPG31A. 
description  of,  300 
factor  content,  239 

(Sec  validation  of  experimental 
printed  tests.) 

Equipment,  Radar, 
early  development,  5 
trends  In  development  of,  53 
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Equipment,  ru (In r.  (See  »/. so  specific 
trainer  or  designation  of  specific 
sot.) 

function  of,  2 7 
oporutlon  of,  29 
radar  observer  equipment,  6 
European  Theater. 

Job  desrript Ion  of  radar  observer's 
tasks  In,  47 
radar  research  In,  210 
Examiner, 
functions  of,  130 
training  of,  08 

Experimental  selection  tests — Appendix 
A.  (See  validation,  selection  tests.) 
Factors  (abilities,  aptitudes), 
definitions,  24 
in  ASV  operation,  211 
in  NDUC  Project's  tests,  215 
predicted  validity  of  factors  for  radar 
observer  training,  40 
Factor  approach  to  selection  test  re¬ 
search,  22 
Factor  testa: 

In  experimental  validation  battery, 
239 

Final  Test  I.  (Sec  Radar  final  test  1.) 
Final  Test  II.  ( Sec  Radar  final  test  II.) 
Final  test  for  AN/APQ-7: 

description  of,  85 
Finger  dexterity,  CM116A: 

correlation  with  success  In  radar  ob¬ 
server  school,  219 
description  of,  293 
factor,  content,  25 
(See  validation  of  uir-crew  classifica¬ 
tion  battery.) 

validity  for  radar  observer  training, 
208,  209 
Flx-tnklng,  30 

Flight  Orientation,  CP528A : 
description  of,  308 
factor  content,  20 

(See  validation  of  experimental 
printed  testa.) 

Form  conversion  speed  test  I  and  II: 
description  of,  212 
factor  loadings  In,  215 
Intercorrelatlona  with  NDRC  project’s 
tests,  215 

Form  detection  test  (NDRC): 
description  of,  212 
factor  loadings  In,  215 
Intercorrelatlona  with  NDRC  project'd 
tests,  215 


Future  radar  observer's  Job.  ( See  Job 
Deserlpi  Ion.) 

< leiirr.nl  information,  CE505H: 
description  of,  285 
factor  content,  25 

validity  for  radar  observer  training, 
2t>8.  2(59 

(Sir  validation  of  aircrew  classifica¬ 
tion  battery.) 

Grades  in  course: 

description  of,  231,  242 
reliability  of,  232,  213 
Instructional  objectives,  are  of,  In  con¬ 
st  rnet  lug  proficiency  measures: 
in  aerial  performance  cheeks,  lit) 

111  bench  set  checks,  100 
hi  supersonic  I  miner  cheeks,  100 
Instrument  Comprehension  I,  CI015I1: 
description  of,  285 
(Sec  validation  of  air  crew  classifica¬ 
tion  battery.) 

1  list rnineiit  Comprehension  II,  CI010B: 
description  of,  287 
factor  content,  20 

(Sec  validation  of  air  crew  classifica¬ 
tion  battery.) 

Intercorrelations  of  proficiency  meas¬ 
ures:  117 

ns  a  validity  criterion,  141 
lntercorrchitloiis  of  selection  tests: 
AF.RD  No.  1  exiierliiieiitni  battery, 
220 

of  NDRC  project's  tests,  215 
Job  analogy: 

approach  to  selection  test  research, 
22,  23 

evaluation  of,  255 
Job  analysis: 
functions  of,  22 
methods  of,  22 
of  ASV  operator’s  task,  211 
of  radar  observer's  task 
in  training,  20 
la  combat,  51 
in  the  future,  54 

Job  description.  ( Sec  Job  analysis.) 

In  training,  20 
In  combat,  47 
in  the  future,  54 
of  radar  observer’s  task,  217 
Langley  Field: 

selected  ns  location  for  l’RI'(R),  14 
Lend  crew  (Pathfinder  Crew)  : 
functions  of,  7 
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Mathematics  A,  C1702F: 
descripl  Ion  of,  287 

(See  validation  of  air-crow  clnsslflcu- 
tlon  battery.) 

validity  for  radar  bombing  error,  2G8, 
200 

validity  for  radar  observer  training, 
208,  2 GO 

Mathematics  B,  CI20CO: 
description  of,  288 
(See  validation  of  air  crew  classifica¬ 
tion  battery.) 

validity  for  radar  bombing  error,  2GS, 

2  GO 

validity  for  radar  observer  training, 
208,  200 

Mechanical  Information,  CI005A. 
description  of,  308 
factor  content,  25,  230 
Indicated  validity  of  factor  content, 
233 

(See  validation  of  experimental 
printed  tests.) 

validity  for  rndur  observer  training, 
208,  200 

Mechanical  Principles,  CI003B. 

correlation  with  success  In  radar  ob¬ 
server  school,  210 
description  of,  288 
In  AKItD  No.  1  battery,  218 
Indicated  validity  of  factor  content, 
258 

(See  validation  of  air  crew  classifica¬ 
tion  battery.) 

validity  for  radar  observer  training, 
208,  209 

Memory  for  Landmarks,  CI510AX2. 
description  of,  309 
factor  content,  25 

Indicated  validity  of  factor  content, 
250 

(See  validation  of  experimental 
printed  test3.) 

Mosaic  method.  (See  Bomb  scoring.) 
Multiple  regression  statistics 
for  bombardier  sample,  252 
for  navigator  sample,  25-1 
for  rights  and  wrongs  scores,  252 
Multiple  correlation.  (See  Multiple  re¬ 
gression  statistics.) 


National  Defense  Krsonrch  Committee 
(NDltC  project). 

ASV  research,  2 11 
description  of  activities,  11 
description  of  tests  developed,  212 
factor  loadings  of  tests  developed, 
215 

lntercorrclations  of  tests  developed, 
215 

personnel  assigned  to,  11 
Navigation, 

aptitude  required  by  radar  observer, 
217 

correlation  of  sta.dne  with  success  In 
radar  observer  school,  210 
Navigation  proficiency  test 

correlations  with  success  in  radar  ob¬ 
server  school,  219 
description  of,  88 

selection  test  for  bombardiers,  224 
Navigation  proficiency  test. 

Form  P5-A,  80 
Form  P5-B,  90 
Navigators. 

requirements  for  rndnr  observer 
school  selection,  224 
Selection  for  ROB  training,  224 
success  In  rndnr  school  compared  to 
bombardiers,  222 
Navigator  stanlne. 

(See  validation  of  air-crew  specialty 
stnnlnes.) 

validity  for  radar  bombing  error,  208, 
209 

validity  for  rndnr  observer  trnlnlng, 
208,  209 

Numerical  operations,  CI702BXL 
as  Job  analogy  test,  240 
correlation  with  scores  in  radnr  ob¬ 
server  school,  219 
description  of,  310 
factor  content,  230 

(See  validation  of  experimental 
printed  tests.) 

validity  for  rndnr  observer  training, 
208,  200 

Orientation  to  landmarks, 
correlation  with  success  in  radar  ob¬ 
server  school,  210 
description  of,  218 
In  AKItD  No.  1  battery,  218 
Oscilloscope  conversion  (NDRO). 
description  of,  234 
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Oscilloscope  lntcrprctallon,  CPS17A. 
correlation  with  success  In  radar  ob¬ 
server  school,  219 
description  of,  2!)S 
in  AERD  No.  1  battery.  218 
in  Headquarters  AAF  battery,  224 
in  Navigator  Project’s  battery,  224 
intercorrelations,  220 
(Sc-o  validation  of  radar  observer  se¬ 
lection  battery.) 

Oscilloscope  reading  (NDRC). 
description  of,  212 
factor  loadings  in,  215 
lntercorrolntions  with  NDRO  Proj¬ 
ect’s  test,  215 
Pacific  Theater. 

job  description  of  radar  observer's 
tasks  in,  40 

Pathfinder  School  of  Eighth  Air  Force, 
activation  of,  7 
selection  requirements,  216 
Pattern  Comprehension,  CF803A. 
description  of,  311 
factor  content,  26 

( See  validation  of  experimental 
printed  tests.) 

Pattern  Identification,  CPS20A. 
correlation  with  success  In  radar  ob¬ 
server  school,  219 
description  of,  311 
in  AERD  No.  1  battery,  218 
(See  validation  of  experimental 
printed  tests.) 

Pattern  Orientation,  CP810A. 
correlation  with  success  In  rndar  ob¬ 
server  school,  219 
description  of,  298 
In  AERD  No,  1  battery,  218 
In  headquarters  AAF  battery,  224 
intercorrelntlons,  220 
(See  validation  of  radar  observer  se¬ 
lection  battery.) 

validity  for  radar  observer  training, 
2GS,  2G9 

Percentiles.  (See  Grades  In  course.) 
Performance  checks.  (See  also  specific 
check.) 

correlations  between  ratings  and  final 
tests,  156 
description  of,  97 

procedure  and  precision  Items  in,  100 
reliability  of,  133 
revisions,  reasons  for,  08 
structure  of,  127 


Performance  checks — Continued, 
list'd  in  radar  course,  95 
validity  of,  138 

Performance,  measurement  of. 
reliability,  133 

Performance,  student,  129 

Photo  bomb  scoring.  (See  llomblng 
scoring.) 

Pilots. 

selection  for  ROl!  training,  224 
stnnlnc  correlation  with  success  in 
radar  observer  schools,  219 

Pilot  stanine.  (Sec  validation  of  air¬ 
crew  specialty  stanlnes.) 
validity  for  radar  observer  trallng,* 
208,  200 

Plot  reading  (NDRC). 
description  of,  213 
factor  loadings  in,  215 
intereorrolatlons  with  NDRO  proj¬ 
ect’s  testa,  215 

Polar  Grid  Coordinate,  Cr8i014. 
ns  Joii  analogy  test,  240 
description  of,  312 
factor  loadings  In,  215 
In  navigator  project’s  battery,  224 
Intereurreintlons  with  NDRO  proj¬ 
ect’s  testa,  215 

validity  for  rmlnr  observer  training, 
2G8,  209 

Position  Orientation,  CP52GA, 
description  of,  313 
factor  content,  210 
(See  validation  of  experimental 
printed  tests.) 

Preference  blank  for  radar  training. 

In  headquarters  AAF  battery,  224 
in  navigator  project’s  bnttery,  22-4 

Printed  proficiency  tests, 
cor  rein  t  ions  with  performance  checks, 
147 

description  of  typical,  CO 
development  of,  CO 
buses  for  revision,  C3 
preliminary  drafts,  GO 
review  and  criticism,  01 
scoring  formula,  02 
standardization  of,  0-4 
test  format,  62 
time  limits,  02 
use  of,  05 

use  of  In  radar  observer  training,  53 

Printed  testa, 
description  of,  06 
development,  00 
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